From 53aef245f9c416624a23b7597b636791b60bb203 Mon Sep 17 00:00:00 2001 From: gsxdsm Date: Fri, 31 Jul 2026 02:17:09 -0700 Subject: [PATCH] test(release): the dry-run safety probe matched a prompt that no longer exists (#3045) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## A release-safety alarm that was firing on wording ``` AssertionError: dry-run must exit before proceed confirmation ``` The probe searched for ``await confirm(`Proceed with release``. The prompt has since become: ```js await confirm(`Proceed with ${CHANNEL} release v${chosenVersion} (build, publish to npm tag '${NPM_DIST_TAG}', tag)?`) ``` so `indexOf` returned **-1**, `dryRunExitIndex < -1` was false, and this has been red on `main` ever since. ## The property itself holds — verified directly, not inferred from a green suite | | offset | | --- | --- | | first `if (DRY_RUN) {` guard, calling `process.exit(0)` | **28808** | | the sole `await confirm(` call site | **44503** | Two dry-run exit guards, one confirmation, exit first. **`pnpm release --dry-run` cannot reach the proceed prompt.** This was never a real safety failure. The probe is narrowed to the stable prefix `await confirm(\`Proceed with `, which still names the one confirmation in the file while surviving the interpolated channel and version. The sentence was never the safety property. ## Why this one mattered more than an ordinary stale probe A stale probe on a *safety* test spends the alarm on cosmetics. Everyone learns the assertion is red for no reason — so a genuine reordering later arrives at an alarm nobody reads. That is a worse outcome than the test not existing. ## Proven to still catch the real regression I injected a `confirm(` call **above** the first dry-run exit in `release.mjs`: ``` confirm injected before the dry-run exit → ℹ pass 5 ℹ fail 1 (dry-run must exit before proceed confirmation) reverted → ℹ pass 6 ℹ fail 0 ``` **No release command was run.** The mutation was local to a scratch copy of `release.mjs`, reverted immediately, and the working tree verified clean — `release.mjs` is untouched by this commit, and the diff is the test file only. ## Fifth and last of the mechanically-fixable suites That closes the mechanical half of the seven red `scripts/__tests__` suites I found: `plugin-authoring-docs` (#3036), `verify-fast` (#3038), `ci-test-shard-timings` (#3040), `engine-vitest-gate-policy` (#3044), and this. **`workflow-reliability-release-check` is the one that is not mechanical** and I am still not touching it: its acceptance map cites 13 test files with **8 missing** and **2 of 5 rows carrying zero surviving evidence**. That needs its owner to decide, per row, whether the coverage moved or was deleted — a repoint would launder the gap (diagnosed on #3036). Five of six looked identical from the failure line. Only that one is unsafe to fix without owning the subject. ## Verification (measured) - this suite — **6 passed / 0 failed** (was 1 failed) - `eslint` — clean Test-only. No changeset. --- scripts/__tests__/release-prompt-gate.test.mjs | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/scripts/__tests__/release-prompt-gate.test.mjs b/scripts/__tests__/release-prompt-gate.test.mjs index 620b248a2c..6d56069fb4 100644 --- a/scripts/__tests__/release-prompt-gate.test.mjs +++ b/scripts/__tests__/release-prompt-gate.test.mjs @@ -63,7 +63,23 @@ test("release script dry-run exits before proceed confirmation and gates ask thr const promptGateIndex = source.indexOf("shouldPromptForVersion({ dryRun: DRY_RUN, autoYes: AUTO_YES, interactive: INTERACTIVE })"); const askIndex = source.indexOf("await ask(`Release version"); const dryRunExitIndex = source.indexOf("if (DRY_RUN) {"); - const confirmIndex = source.indexOf("await confirm(`Proceed with release"); + /* + FNXC:ReleaseSafety 2026-07-31-21:40: + Probe the STABLE prefix, not the whole sentence — the interpolated part is not the safety property. + + This read `await confirm(\`Proceed with release`, and the prompt has since become + `Proceed with ${CHANNEL} release v${chosenVersion} (...)`. `indexOf` returned -1, the comparison + `dryRunExitIndex < -1` was false, and this assertion has been red on `main` ever since — reporting a + release-safety failure that did not exist. + + The property itself still holds and was verified directly rather than inferred from the green: + the first `if (DRY_RUN) {` guard sits at 28808 and calls `process.exit(0)`; the sole `await confirm(` + call site is at 44503. Two dry-run exit guards, one confirmation, and the exit precedes it. + + A stale probe on a SAFETY test is worse than on an ordinary one: it spends the alarm on wording, so + a genuine reordering later arrives at an assertion everyone has learned is red for cosmetic reasons. + */ + const confirmIndex = source.indexOf("await confirm(`Proceed with "); assert.notEqual(promptGateIndex, -1, "release.mjs should use the pure prompt gate"); assert.notEqual(askIndex, -1, "release.mjs should still support version prompts");