Nate Swenson

You cannot feature-detect a refusal

Release · No. 113 · 10 min read ·v0.2.0

Shipped

The preflight report gained an On dev column, printed immediately beside On main, and the minimum supported version of the tool underneath it went from 0.4.0 to 0.6.0. Those two lines are the same fix approached from opposite ends. One puts a fact in front of the operator early enough to act on; the other makes sure the machinery behind it will actually stop when that fact is true.

The second half is the part I keep thinking about, because it runs against advice I would normally give without hesitating: test for the capability, not the version. That advice quietly assumes the capability leaves a trace.

Put the disagreement in the table they read first

The underlying bug was that a release could be cut against whatever version sat on main while a higher version sat on dev, unread. The dependency fixed that by refusing. But a refusal that only speaks at the moment you pull the trigger is a refusal you meet at the worst possible time, after you have already decided to release and told somebody you were doing it.

So the fact moved forward, into the report you read before deciding anything.

// The report an operator reads BEFORE deciding to release anything. Its job is
// to put every fact that can block a cut into the same table, so nobody meets a
// blocker for the first time at the moment they trigger it.
export function preflight(components) {
  const rows = components.map((c) => ({
    Component: c.name,
    'Last tag': c.lastTag ?? '—',
    'On main': c.versionOnMain ?? '—',
    // The column this release added. `On main` and `On dev` disagreeing is the
    // fact a cut used to act on silently, so it belongs next to `On main`
    // rather than three commands away.
    'On dev': c.versionOnDev ?? '—',
    Blockers: (c.blockers ?? []).map((b) => b.id).join(', ') || 'none',
  }));
  return rows;
}

export function renderTable(rows) {
  if (rows.length === 0) return '(nothing to report)';
  const cols = Object.keys(rows[0]);
  const width = (c) =>
    Math.max(c.length, ...rows.map((r) => String(r[c]).length));
  const line = (cells) =>
    '| ' + cols.map((c, i) => String(cells[i]).padEnd(width(c))).join(' | ') + ' |';
  return [
    line(cols),
    '|' + cols.map((c) => '-'.repeat(width(c) + 2)).join('|') + '|',
    ...rows.map((r) => line(cols.map((c) => r[c]))),
  ].join('\n');
}

Column placement is doing real work here. On dev sits immediately after On main because the two disagreeing is the finding, and two numbers only read as a contradiction when they are adjacent. Put one of them in a different command and the operator has to already suspect the problem to go looking.

The probe that proves nothing

Now the interesting half. If your tool depends on another tool’s guard, how do you know the guard is there?

The standard answer is feature detection. MDN puts the reasoning plainly: feature detection means “working out whether a browser supports a certain block of code, and running different code depending on whether it does”, and it warns against the alternative of sniffing versions, which it calls a terrible practice that should be discouraged at all costs. I agree with that in almost every case. if ("geolocation" in navigator) is better than a version table, because the feature is a thing you can reach out and touch.

A refusal is not. Here are two versions of a dependency that differ only in whether the dangerous function honours the fact both of them report:

// Two stand-in versions of the dependency this tool drives. They differ ONLY in
// whether `cut` acts on the fact, which is the guarantee we actually rely on.
const AMBIGUOUS_NOTE = 'main and dev carry different versions';

function readStatus({ name, versionOnMain, versionOnDev }) {
  const devAhead = versionOnDev !== versionOnMain ? { version: versionOnDev } : null;
  return {
    name,
    versionOnMain,
    versionOnDev,
    devAhead,
    blockers: devAhead ? [{ id: 'dev-ahead-of-main', detail: AMBIGUOUS_NOTE }] : [],
  };
}

// Reports the fact. Does NOT refuse on it.
export const reportsOnly = {
  version: '0.5.0',
  readStatus,
  cut(status) {
    return { ok: true, tagged: `v${status.versionOnMain}` };
  },
};

// Reports the fact AND refuses to act while it is set.
export const refuses = {
  version: '0.6.0',
  readStatus,
  cut(status) {
    if (status.devAhead) {
      return { ok: false, error: `ambiguous: ${AMBIGUOUS_NOTE}, refusing to guess` };
    }
    return { ok: true, tagged: `v${status.versionOnMain}` };
  },
};

Probe either one and you get the same answer. devAhead is present in both. blockers has one entry in both. Every field you could reasonably test for exists on the version that will happily tag the wrong thing.

That is the general shape and it is worth naming: detecting a data field does not prove the behaviour that consumes it. Adding a field and acting on it are separate changes, and only the first one is observable. The only honest probe for “does cut refuse?” is to hand cut an ambiguous status and see, which means performing the operation you were trying to protect.

Check the floor yourself, because declaring it does not

If the guarantee is invisible, the version number is the only signal left. The obvious move is to declare it as a dependency constraint and stop thinking about it, and for a library that is usually right. For a tool you shell out to, it is not: npm’s own documentation says the engines field is advisory only unless the user has set the engine-strict config flag, producing warnings rather than refusals. A declared floor and an enforced floor are different things.

So check it in your own process, at startup, before anything can happen.

// The floor check. Runs in your process, at startup, and says what the floor
// buys rather than just quoting two numbers at the operator.
export function cmpSemver(a, b) {
  const pa = String(a).split('.').map(Number);
  const pb = String(b).split('.').map(Number);
  for (let i = 0; i < 3; i++) {
    if ((pa[i] ?? 0) !== (pb[i] ?? 0)) return (pa[i] ?? 0) - (pb[i] ?? 0);
  }
  return 0;
}

export const MIN_DEP = '0.6.0';

// `buys` is the important argument. A floor whose message is "requires >= 0.6.0"
// teaches the reader nothing and invites them to lower it; a floor that names
// the guarantee explains why lowering it is not a config change.
export function requireFloor(actual, min = MIN_DEP, buys = 'the refusal to release an ambiguous target') {
  if (cmpSemver(actual, min) >= 0) return { ok: true, actual };
  return {
    ok: false,
    error:
      `needs >= ${min}, found ${actual}. ${min} is where ${buys} landed. ` +
      `An older version returns a status of the same shape and silently ` +
      `lacks it, so this cannot be detected from the response.`,
  };
}

The buys argument is a small thing that changes how the floor survives. A message reading “requires >= 0.6.0” reads as bureaucracy, and the next person who hits it on a machine with 0.5.0 installed will try lowering the constant to see if it works. It will appear to work, because the missing behaviour is a refusal that does not fire on any of the happy paths they test. Naming the guarantee in the error is what stops that.

Note that cmpSemver parses rather than string-compares. Semantic Versioning requires numeric comparison of major, minor and patch, and lexicographic order puts 0.10.0 below 0.9.0, which in a floor check means silently accepting a version far below your minimum.

Run it

import assert from 'node:assert/strict';
import { preflight, renderTable } from './preflight.mjs';
import { reportsOnly, refuses } from './deps.mjs';
import { requireFloor } from './floor.mjs';

const COMPONENTS = [
  { name: 'widget', lastTag: '0.1.0', versionOnMain: '0.2.1', versionOnDev: '0.3.0' },
  { name: 'gadget', lastTag: '1.4.0', versionOnMain: '1.4.0', versionOnDev: '1.4.0' },
];

// 1. The table an operator reads first, with both branch columns side by side.
const withStatus = COMPONENTS.map((c) => ({ ...c, ...refuses.readStatus(c) }));
console.log(renderTable(preflight(withStatus)));

// 2. Now the probe. Both dependency versions expose the same field.
const ambiguous = COMPONENTS[0];
for (const dep of [reportsOnly, refuses]) {
  const status = dep.readStatus(ambiguous);
  console.log(
    `\n${dep.version}: hasOwn(status,'devAhead') = ${Object.hasOwn(status, 'devAhead')}` +
    ` | blockers = ${status.blockers.length}`
  );
}

// The probe passes on both, so it proves nothing about what cut() will do.
assert.equal(
  Object.hasOwn(reportsOnly.readStatus(ambiguous), 'devAhead'),
  Object.hasOwn(refuses.readStatus(ambiguous), 'devAhead')
);

// 3. The only thing that distinguishes them is running the irreversible call.
console.log('\nwhat cut() actually does with the same input:');
for (const dep of [reportsOnly, refuses]) {
  const result = dep.cut(dep.readStatus(ambiguous));
  console.log(`  ${dep.version}:`, JSON.stringify(result));
}
assert.equal(reportsOnly.cut(reportsOnly.readStatus(ambiguous)).ok, true);
assert.equal(refuses.cut(refuses.readStatus(ambiguous)).ok, false);

// 4. So check the version instead, at startup, before any of this runs.
console.log('\nfloor check:');
for (const dep of [reportsOnly, refuses]) {
  const gate = requireFloor(dep.version);
  console.log(`  ${dep.version}: ${gate.ok ? 'ok' : gate.error}`);
}
assert.equal(requireFloor(reportsOnly.version).ok, false);
assert.equal(requireFloor(refuses.version).ok, true);

console.log('\nall checks passed');

Save the three blocks as preflight.mjs, deps.mjs and floor.mjs, this one as check.mjs, and run node check.mjs. It uses nothing but Node’s own assert module. Here is what it printed for me:

| Component | Last tag | On main | On dev | Blockers          |
|-----------|----------|---------|--------|-------------------|
| widget    | 0.1.0    | 0.2.1   | 0.3.0  | dev-ahead-of-main |
| gadget    | 1.4.0    | 1.4.0   | 1.4.0  | none              |

0.5.0: hasOwn(status,'devAhead') = true | blockers = 1

0.6.0: hasOwn(status,'devAhead') = true | blockers = 1

what cut() actually does with the same input:
  0.5.0: {"ok":true,"tagged":"v0.2.1"}
  0.6.0: {"ok":false,"error":"ambiguous: main and dev carry different versions, refusing to guess"}

floor check:
  0.5.0: needs >= 0.6.0, found 0.5.0. 0.6.0 is where the refusal to release an ambiguous target landed. An older version returns a status of the same shape and silently lacks it, so this cannot be detected from the response.
  0.6.0: ok

all checks passed

The two middle sections are the whole argument sitting next to each other. The probe reports identically for both versions. The operation reports ok: true against one and refuses against the other. If your only evidence is the first block, you have learned nothing about the second.

Gotchas

A floor you raise without saying why gets lowered. The constant is the easy part; the reason it moved is the part that has to travel with it. Put the guarantee in the error string, and put the same sentence in the changelog entry that raises it. Somebody hitting this on a machine with the old version will test whether lowering it breaks anything, and it will not break anything they can see, because the thing they removed only fires on a case they are not in.

Declaring a dependency floor is not enforcing one. engines warns by default. A peer range can be satisfied by a hoisted copy you did not expect. If the tool is invoked as a subprocess rather than imported, package metadata does not constrain it at all: the binary on PATH is whatever is on PATH. Read the version from the thing you are actually going to run.

Bundle the version check with the version report. When you shell out, ask the tool for its version and remember where you found it. Reporting “found 0.5.0 at /usr/local/bin/tool” turns a confusing failure into a one-step fix, because the most common cause of a too-old dependency is a stale global install shadowing the one you thought you were running.

A blocker that only appears at the point of action is a blocker discovered too late. This is the ordering half of the same lesson. If a condition can stop an operation, it belongs in whatever report people read while deciding to perform that operation, not only in the guard that fires when they do. The guard still has to exist; it is the last line, not the first.

Do not let the fix for an overclaim be a softer claim. A rationale in this release said its fixtures covered “the grouper and the state machine” when no assertion in it read state at all, and all three fixtures carried the same value. The correction was to name only what is genuinely covered and point at where the rest is actually pinned. Documentation that describes coverage you do not have is worse than documentation that describes less, because it stops anybody from going looking.

Sources

Changelog

  • fix(release): surface dev-ahead-of-main before cut, not only after it refuses (0.2.0) (#179) (29efa06)