Nate Swenson

Own part of a file without owning the file

Press · No. 084 · 16 min read ·v0.1.0

Shipped

press 0.1.0 replaced eight hand-ported copies of one brand across four repositories with a single tokens.json and a CI drift gate. Each copy had documented its own lineage in a header comment, which is a copy chain rather than a system: five different names for the same orange, and three token groups that were genuinely shared but declared nowhere. The release declared all eight as targets, migrated the five that live in the same monorepo, and added press check to fail the build when any of them drifts. Those five span Python, CSS and Markdown.

The interesting part isn’t the token file. It’s that none of those consumers is generated. Each one is a hand-written stylesheet or renderer with a small shared block inside it, and press owns only the block. This guide builds that mechanism: a marked region you can splice into a file you don’t own, plus the gate that keeps it honest.

Why a region and not a file

The obvious design is to generate the whole consumer file from the tokens. It’s simpler to write and it’s wrong for most codebases, because the shared part is usually small and the rest is medium-specific work you’d be destroying on every sync. In press the shared core is around 80 lines while a single consumer’s stylesheet runs past 250, and a whole-file sync had already erased a hand-written avatar footer once before.

Marked regions are well-trodden. Ansible’s blockinfile inserts and updates a block between customizable marker lines, and its docs are blunt about the failure mode: a custom marker that omits the {mark} placeholder gets the block re-inserted on every run instead of updated. Go standardized the other half, the warning to humans. Generated Go source is expected to carry a line matching ^// Code generated .* DO NOT EDIT\.$, and per the go generate docs, “This line must appear before the first non-comment, non-blank text in the file.”

We want both properties, plus one more: the marker should carry a receipt, so a diff tells you which release last wrote the block and a hand-edit is detectable without re-running the generator.

Define the tokens and the consumers

Everything below is Node’s standard library with no dependencies, so an empty directory and Node 18 or newer is the whole setup. Start with the source of truth. Keep it dumb; it’s data, not code. The Design Tokens Format Module standardizes a richer JSON shape for exactly this if you want cross-tool interoperability later, but a flat object is enough to build on.

{
  "colors": {
    "paper": "#F5F1E8",
    "ink": "#1A1A1A",
    "accent": "#FF6B35"
  },
  "fonts": {
    "display": "Inter, system-ui, sans-serif"
  }
}

Then declare the consumers. A target is one region, in one file, in one repo. This registry is what makes a file the tool’s business; anything not listed here is invisible to the gate, which is the whole failure mode you’re closing.

{
  "targets": [
    {
      "id": "site-css",
      "path": "site/brand.css",
      "region": "vars",
      "syntax": "css",
      "emitter": "css-vars",
      "params": { "selector": ":root", "alias": { "accent": "sig" } },
      "init": { "replaceFrom": "^:root \\{$", "replaceTo": "^\\}$" }
    },
    {
      "id": "report-py",
      "path": "report/brand.py",
      "region": "vars",
      "syntax": "python",
      "emitter": "python-dict",
      "params": { "name": "THEME" },
      "init": { "replaceFrom": "^THEME = \\{$", "replaceTo": "^\\}$" }
    }
  ]
}

The alias map matters more than it looks. Consumers will already have their own names for a value, and forcing every repo to rename its variables on adoption day is how a migration stalls. Let the registry translate.

Build the region protocol

Save this as region.mjs. It knows four things: how to write a marker in a given comment syntax, how to find a region, how to replace one, and how to create one for the first time.

import { createHash } from 'node:crypto';

// One comment syntax per file type you need to write into.
export const SYNTAXES = {
  css: { prefix: '/* ', suffix: ' */' },
  python: { prefix: '# ', suffix: '' },
};

const BANNER = 'GENERATED, do not edit';

export class RegionError extends Error {}

const syntaxOf = (name) => {
  const s = SYNTAXES[name];
  if (!s) throw new RegionError(`unknown syntax "${name}"`);
  return s;
};

const wrap = (s, text) => `${s.prefix}${text}${s.suffix}`;

// The inner text of a comment line, or null if this line is not one.
function unwrap(s, line) {
  const t = line.replace(/\s+$/, '');
  if (!t.startsWith(s.prefix)) return null;
  if (s.suffix && !t.endsWith(s.suffix)) return null;
  return t.slice(s.prefix.length, t.length - s.suffix.length);
}

export function bodyHash(body) {
  const normalized = body.replace(/\r\n/g, '\n').replace(/\s+$/, '');
  return createHash('sha256').update(normalized, 'utf8').digest('hex').slice(0, 12);
}

export const startMarker = (syntax, region, version, hash) =>
  wrap(syntaxOf(syntax), `>>> tokens:${region} v${version} sha256:${hash} ${BANNER}`);

export const endMarker = (syntax, region) =>
  wrap(syntaxOf(syntax), `<<< tokens:${region}`);

const escapeRe = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');

export function findRegion(text, region, syntax) {
  const s = syntaxOf(syntax);
  const lines = text.split('\n');
  const openRe = new RegExp(`^>>> tokens:${escapeRe(region)}(?:\\s|$)`);
  const closeRe = new RegExp(`^<<< tokens:${escapeRe(region)}\\s*$`);

  let start = -1;
  for (let i = 0; i < lines.length; i += 1) {
    const inner = unwrap(s, lines[i]);
    if (inner !== null && openRe.test(inner)) {
      if (start !== -1) throw new RegionError(`region "${region}" opens twice`);
      start = i;
    }
  }
  if (start === -1) return null;

  let end = -1;
  for (let i = start + 1; i < lines.length; i += 1) {
    const inner = unwrap(s, lines[i]);
    if (inner !== null && closeRe.test(inner)) { end = i; break; }
  }
  if (end === -1) throw new RegionError(`region "${region}" opens but never closes`);

  const receipt = /\bv(\S+)\s+sha256:([0-9a-f]+)/.exec(unwrap(s, lines[start]));
  return {
    startLine: start,
    endLine: end,
    body: lines.slice(start + 1, end).join('\n'),
    version: receipt ? receipt[1] : null,
    hash: receipt ? receipt[2] : null,
  };
}

export function renderRegion(region, syntax, body, version) {
  const trimmed = body.replace(/\s+$/, '');
  return [
    startMarker(syntax, region, version, bodyHash(trimmed)),
    trimmed,
    endMarker(syntax, region),
  ].join('\n');
}

// Replace an existing region. Refuses to guess if there isn't one.
export function spliceRegion(text, region, syntax, body, version) {
  const found = findRegion(text, region, syntax);
  if (!found) throw new RegionError(`file has no tokens:${region} region; run with --init first`);
  const lines = text.split('\n');
  lines.splice(
    found.startLine,
    found.endLine - found.startLine + 1,
    ...renderRegion(region, syntax, body, version).split('\n'),
  );
  return lines.join('\n');
}

// First-time insertion. `replaceFrom`/`replaceTo` name the first and last line
// of the hand-written block the region takes over, so the old copy is swallowed
// instead of left behind to drift.
export function initRegion(text, region, syntax, body, version, anchor = {}) {
  if (findRegion(text, region, syntax)) {
    throw new RegionError(`file already has a tokens:${region} region`);
  }
  const block = renderRegion(region, syntax, body, version);
  const { replaceFrom, replaceTo } = anchor;
  if (!replaceFrom) return `${text.replace(/\s+$/, '')}\n\n${block}\n`;

  const lines = text.split('\n');
  const from = lines.findIndex((l) => new RegExp(replaceFrom).test(l));
  if (from === -1) throw new RegionError(`anchor replaceFrom /${replaceFrom}/ matched no line`);
  const toRe = new RegExp(replaceTo ?? replaceFrom);
  let to = -1;
  for (let i = from; i < lines.length; i += 1) {
    if (toRe.test(lines[i]) && (i > from || !replaceTo)) { to = i; break; }
  }
  if (to === -1) throw new RegionError(`anchor replaceTo /${replaceTo}/ matched no line`);
  lines.splice(from, to - from + 1, ...block.split('\n'));
  return lines.join('\n');
}

Three decisions in there are worth pulling out. spliceRegion throws rather than appending when the region is absent, because silently creating a second copy is precisely the Ansible failure mode. The init anchor swallows the legacy hand-written block in the same operation that inserts the region, so adoption never leaves a duplicate definition behind. And the hash is over a whitespace-normalized body, so a stray trailing newline doesn’t read as a change.

Emit the body from the tokens

An emitter turns tokens into the text that goes between the markers. Save this as emit.mjs. Every emitter is a pure function of (tokens, params), which is the property the gate depends on later.

export class EmitError extends Error {}

// Preferred order for output. A SORT KEY, never a filter: anything not named
// here sorts last instead of being dropped, so a token added to tokens.json
// always reaches every consumer.
const ORDER = ['paper', 'ink', 'accent'];
const rank = (k) => (ORDER.indexOf(k) === -1 ? ORDER.length : ORDER.indexOf(k));

const flatten = (tokens) =>
  Object.entries({ ...tokens.colors, ...tokens.fonts }).sort(
    ([a], [b]) => rank(a) - rank(b) || a.localeCompare(b),
  );

function cssVars(tokens, params) {
  const selector = params.selector ?? ':root';
  const alias = params.alias ?? {};
  const body = flatten(tokens).map(([k, v]) => `  --${alias[k] ?? k}: ${v};`);
  return [`${selector} {`, ...body, '}'].join('\n');
}

function pythonDict(tokens, params) {
  const name = params.name ?? 'THEME';
  const body = flatten(tokens).map(([k, v]) => `    ${JSON.stringify(k)}: ${JSON.stringify(v)},`);
  return [`${name} = {`, ...body, '}'].join('\n');
}

export const EMITTERS = { 'css-vars': cssVars, 'python-dict': pythonDict };

// Same inputs, same bytes; that is what lets `check` re-derive what should be
// on disk.
export function emitBody(tokens, emitter, params = {}) {
  const fn = EMITTERS[emitter];
  if (!fn) {
    throw new EmitError(
      `unknown emitter "${emitter}" (expected one of: ${Object.keys(EMITTERS).join(', ')})`,
    );
  }
  return fn(tokens, params);
}

The gate: re-derive and compare

Now the part that earns the whole design. check regenerates what each region should contain and compares it to what’s on disk, and it has to fail on more than the obvious case. Save this as check.mjs.

import { existsSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { emitBody } from './emit.mjs';
import { findRegion, RegionError } from './region.mjs';

export const EXPLAIN = {
  ok: 'in sync',
  drift: 'region content differs from what the tokens produce',
  'stale-version': 'values are correct but the region records an older release',
  missing: 'file has no region; the generated block was removed',
  absent: 'declared file does not exist at this path',
  corrupt: 'region markers are malformed',
};

export function checkTarget(target, root, tokens, version) {
  const path = join(root, target.path);
  if (!existsSync(path)) return { target, path, status: 'absent' };
  const text = readFileSync(path, 'utf8');

  let found;
  try {
    found = findRegion(text, target.region, target.syntax);
  } catch (err) {
    if (err instanceof RegionError) return { target, path, status: 'corrupt', detail: err.message };
    throw err;
  }
  if (!found) return { target, path, status: 'missing' };

  const expected = emitBody(tokens, target.emitter, target.params).replace(/\s+$/, '');
  const actual = found.body.replace(/\s+$/, '');
  if (expected !== actual) {
    return { target, path, status: 'drift', diff: lineDiff(expected, actual) };
  }
  // The values match, but the region must also record the release it belongs
  // to. Skip this and the version in the marker is decoration.
  if (found.version !== version) {
    return { target, path, status: 'stale-version', writtenBy: found.version };
  }
  return { target, path, status: 'ok', writtenBy: found.version };
}

export function checkAll({ tokens, targets, root, version }) {
  const selected = targets.filter((t) => existsSync(join(root, t.path)));
  const results = selected.map((t) => checkTarget(t, root, tokens, version));
  const failures = results.filter((r) => r.status !== 'ok');
  return {
    results,
    failures,
    // A run that resolved ZERO targets is itself a failure. A glob that
    // quietly matches nothing must go red; that is how a gate turns decorative.
    empty: selected.length === 0,
    ok: selected.length > 0 && failures.length === 0,
  };
}

// Elide the common prefix and suffix so a one-token change reads as one line.
export function lineDiff(expected, actual) {
  const a = expected.split('\n');
  const b = actual.split('\n');
  let head = 0;
  while (head < a.length && head < b.length && a[head] === b[head]) head += 1;
  let tail = 0;
  while (
    tail < a.length - head && tail < b.length - head &&
    a[a.length - 1 - tail] === b[b.length - 1 - tail]
  ) tail += 1;
  const out = [];
  if (head) out.push(`  ... ${head} identical line${head === 1 ? '' : 's'}`);
  for (const l of b.slice(head, b.length - tail)) out.push(`- ${l}`);
  for (const l of a.slice(head, a.length - tail)) out.push(`+ ${l}`);
  if (tail) out.push(`  ... ${tail} identical line${tail === 1 ? '' : 's'}`);
  return out.join('\n');
}

missing and empty are the two statuses people leave out, and they’re the two that decide whether the gate is real. A consumer that dropped its markers but kept the values would otherwise pass forever while checking nothing, and a registry whose paths all stopped resolving would report a clean run over zero files.

The exit code follows Terraform’s convention for the same reason. terraform plan -detailed-exitcode distinguishes 0 for an empty diff, 1 for an error, and 2 for changes present, which lets a pipeline tell “your config is broken” apart from “your world drifted.” Wire the CLI up as press.mjs:

#!/usr/bin/env node
import { readFileSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { emitBody } from './emit.mjs';
import { initRegion, spliceRegion } from './region.mjs';
import { checkAll, EXPLAIN } from './check.mjs';

const VERSION = '0.1.0';
const root = process.cwd();
const tokens = JSON.parse(readFileSync(join(root, 'tokens.json'), 'utf8'));
const targets = JSON.parse(readFileSync(join(root, 'targets.json'), 'utf8')).targets;

const [cmd, ...rest] = process.argv.slice(2);
const init = rest.includes('--init');

if (cmd === 'emit') {
  for (const t of targets) {
    const path = join(root, t.path);
    const before = readFileSync(path, 'utf8');
    const body = emitBody(tokens, t.emitter, t.params);
    const after = init
      ? initRegion(before, t.region, t.syntax, body, VERSION, t.init)
      : spliceRegion(before, t.region, t.syntax, body, VERSION);
    writeFileSync(path, after);
    console.log(`${after === before ? 'unchanged' : 'wrote'}  ${t.id}  ${t.path}`);
  }
} else if (cmd === 'check') {
  const { results, ok, empty } = checkAll({ tokens, targets, root, version: VERSION });
  for (const r of results) {
    console.log(`${r.status === 'ok' ? 'ok  ' : 'FAIL'}  ${r.target.id}  ${EXPLAIN[r.status]}`);
    if (r.diff) console.log(r.diff.replace(/^/gm, '        '));
  }
  if (empty) console.log('FAIL  no targets resolved in this checkout');
  process.exit(ok ? 0 : 2);
} else {
  console.error('usage: press.mjs <emit|check> [--init]');
  process.exit(1);
}

Run it

Create the two consumer files with their existing hand-written copies still in place, so you can watch the migration take them over. Note the parts of each file that have nothing to do with tokens; those are what you’re protecting.

mkdir -p site report
cat > site/brand.css <<'EOF'
/* The site's stylesheet. Mostly hand-written, medium-specific work. */
:root {
  --paper: #F5F1E8;
  --ink: #1A1A1A;
  --sig: #FF6B35;
}

.masthead {
  background: var(--paper);
  color: var(--ink);
  border-bottom: 2px solid var(--sig);
}

/* Hand-written and load-bearing: a whole-file sync would erase this. */
.masthead__avatar {
  width: 44px;
  border-radius: 50%;
}
EOF
cat > report/brand.py <<'EOF'
"""The report renderer's theme. Hand-written except for the token block."""

THEME = {
    "paper": "#F5F1E8",
    "ink": "#1A1A1A",
    "accent": "#FF6B35",
}


def page_margins(paper_size):
    """Medium-specific work that has no business in a shared token file."""
    return {"letter": (54, 54), "a4": (57, 51)}[paper_size]
EOF

node press.mjs emit --init
cat site/brand.css

The migration swallows each legacy block and leaves everything else alone:

wrote  site-css  site/brand.css
wrote  report-py  report/brand.py
/* The site's stylesheet. Mostly hand-written, medium-specific work. */
/* >>> tokens:vars v0.1.0 sha256:de15320eccba GENERATED, do not edit */
:root {
  --paper: #F5F1E8;
  --ink: #1A1A1A;
  --sig: #FF6B35;
  --display: Inter, system-ui, sans-serif;
}
/* <<< tokens:vars */

.masthead {
  background: var(--paper);
  color: var(--ink);
  border-bottom: 2px solid var(--sig);
}

/* Hand-written and load-bearing: a whole-file sync would erase this. */
.masthead__avatar {
  width: 44px;
  border-radius: 50%;
}

The avatar rule survived, and --display arrived even though the hand-written copy never had it. Now change a token without re-emitting, which is the thing you actually want CI to catch:

sed -i '' 's/#FF6B35/#E8590C/' tokens.json   # GNU sed: drop the ''
node press.mjs check; echo "exit: $?"

FAIL  site-css  region content differs from what the tokens produce
          ... 3 identical lines
        -   --sig: #FF6B35;
        +   --sig: #E8590C;
          ... 2 identical lines
FAIL  report-py  region content differs from what the tokens produce
          ... 3 identical lines
        -     "accent": "#FF6B35",
        +     "accent": "#E8590C",
          ... 2 identical lines
exit: 2

One token moved and both consumers reported it, in their own syntax, pointing at the exact line. node press.mjs emit puts them back in sync and check returns 0. Then try the two failures that only exist because you wrote them: strip the marker comments out of site/brand.css while leaving the values, and check reports missing rather than passing. Run the same CLI in a directory where none of the target paths exist, and it reports that zero targets resolved instead of a clean green run.

Those two statuses cost about six lines between them, and without either one the gate can report a clean run while inspecting nothing.

Gotchas

A sort key that’s quietly a filter. You’ll want deterministic output ordering, and the natural way to get it is to iterate a list of known token names. The trap is that the list then decides membership too. Symptom: you add a value to the token file, no consumer ever receives it, and the checker still reports green because the emitter and the gate agree on the same wrong answer. The escape is to rank unknown names last rather than dropping them, and to pin a test asserting that a token absent from the order list still shows up in the output. This one was caught during the press migration itself.

Generating the whole file eats hand-written work. Whole-file generation looks cleaner until a consumer file has anything personal in it. Symptom: a hand-written block vanishes during a routine sync and nobody notices until it’s shipped, which is how a personal avatar footer disappeared from a card stylesheet before press existed. The escape is the region: own the block, never the file, and make the tool throw rather than append when the region is absent.

The version in the marker is decoration until the checker compares it. Writing a version into the start marker feels like it makes the region self-describing, but if check compares only the body bytes then a region whose values happen to still be correct will pass at any age. This bit press directly: in-repo regions sat recording v0.1.0 through six subsequent releases with a green tick. The fix in 0.7.1 was to give a version mismatch its own status, stale-version, separate from drift, so it reports and prints the re-emit command instead of silently passing.

File presence is not repository identity. Selecting targets by “does this path exist under the repo root” works fine until a target’s path is something every repository has. Symptom: press added README.md targets in 0.7.0, and because README.md exists everywhere, those targets selected inside any checkout and would have been compared against a completely unrelated file. The escape is to read the checkout’s own origin remote and match it against the target’s declared repo, falling back to presence only when there is no remote, which is the case inside a test’s temp directory.

Sources

Changelog

  • feat(press): one brand system, generated into every consumer (0.1.0) (#115) (3f3c26d)