FN-9147: synchronize script test baselines

Align script-test expectations and documentation with the current validator and dashboard test inventory. - Add the pre-JSON anchor validator to the verify-fast baseline. - Ratchet documented pretest validators against canonical order and membership. - Remove deleted dashboard tests from timing data. Files changed: docs/testing.md | 22 +++++++++- scripts/__tests__/verify-fast.test.mjs | 76 +++++++++++++++++++++++++++++++++- scripts/test-timings.json | 2 - 3 files changed, 95 insertions(+), 5 deletions(-) Fusion-Task-Id: FN-9147 Fusion-Task-Lineage: fdb33c60-e019-402b-b91b-ba0f0f971350 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
2026-08-19 05:23:41 -07:00
parent ed2cbd08a1
commit 91daa37162
3 changed files with 95 additions and 5 deletions
--- a/docs/testing.md
+++ b/docs/testing.md
@@ -71,7 +71,27 @@ pnpm verify:workspace  # deep opt-in verification: lint -> test:full -> build (N
 <!-- FNXC:TestInfrastructure 2026-06-25-00:00: verify:fast is the opt-in test-free verification path. docs/testing.md observes the broad test gate caught no recalled real bugs while consuming ~70% of shipping time in flake triage; typecheck+build+boot-smoke gives deterministic, flake-free signal without running tests. It changes no default — pnpm test, the merge gate, and CI are untouched; the full suite stays available and runs non-blocking. -->
 <!-- FNXC:TestInfrastructure 2026-06-26-00:49: verify:fast must bootstrap missing workspace dist artifacts and build @runfusion/fusion even when the CLI package is not in the changed-package set because package builds and the boot smoke invoke source-checkout wrappers that require dist outputs in fresh worktrees. -->
 <!-- FNXC:TestInfrastructure 2026-07-22-12:00: Cheap deterministic policy gates must fail before verify:fast's expensive work. Read canonical package.json pretest commands and invoke their validator entry points directly so test-free verification and the merge gate cannot drift. -->
-`pnpm verify:fast` (`scripts/verify-fast.mjs`) is the recommended **test-free verification** command. It first runs the canonical, read-only static validators from root `pretest` — `check-no-nohup`, `check-no-kill-4040`, `check-no-getdatabase`, `check-prerebase-inert`, `check-cli-runtime-routing`, `check-no-node-only-core-imports-in-dashboard`, `check-pi-versions-pinned`, `check-workspace-package-graph`, `check-no-test-timeout-appeasement`, `check-changeset-format`, `check-routes-modular`, and `check-runtime-skill-loader-drift` (which enforces the Claude/Grok runtime skill loaders' clean rename-diff) — then bootstraps missing/stale workspace dist artifacts, runs **typecheck + build scoped to the changed packages** (reusing the same git-diff / changed-package resolution as `pnpm test`), always builds the `@runfusion/fusion` CLI package required by the source-checkout boot smoke, and runs the existing **boot smoke** once. The static phase invokes each existing validator entry point without update flags, is bounded and fail-fast, and runs **no Vitest or test lane**. It gives deterministic, flake-free signal in seconds, so it is a sound project `testCommand`/verification command when you want non-test verification. With no affected package (root/docs-only diff) it runs static checks, artifact bootstrap, the CLI prerequisite build, and boot smoke. Each step is bounded by the shared `runWithWatchdog` (class `changed`) so a hang fails fast, and it exits nonzero on the first failing step. This is purely additive: it does not change `pnpm test`, the merge gate, or CI, and the full suite stays available (`pnpm test:full`, non-blocking on push to main).
+`pnpm verify:fast` (`scripts/verify-fast.mjs`) is the recommended **test-free verification** command. It first runs the canonical, read-only static validators from root `pretest`:
+
+<!-- pretest-validators:start -->
+- `check-no-nohup`
+- `check-no-cwd-relative-dashboard-test-reads`
+- `check-no-kill-4040`
+- `check-no-getdatabase`
+- `check-prerebase-inert`
+- `check-capacity-pool-id`
+- `check-cli-runtime-routing`
+- `check-no-node-only-core-imports-in-dashboard`
+- `check-pi-versions-pinned`
+- `check-workspace-package-graph`
+- `check-no-test-timeout-appeasement`
+- `check-changeset-format`
+- `check-pre-json-anchor`
+- `check-routes-modular`
+- `check-runtime-skill-loader-drift`
+<!-- pretest-validators:end -->
+
+`check-runtime-skill-loader-drift` enforces the Claude/Grok runtime skill loaders' clean rename-diff. It then bootstraps missing/stale workspace dist artifacts, runs **typecheck + build scoped to the changed packages** (reusing the same git-diff / changed-package resolution as `pnpm test`), always builds the `@runfusion/fusion` CLI package required by the source-checkout boot smoke, and runs the existing **boot smoke** once. The static phase invokes each existing validator entry point without update flags, is bounded and fail-fast, and runs **no Vitest or test lane**. It gives deterministic, flake-free signal in seconds, so it is a sound project `testCommand`/verification command when you want non-test verification. With no affected package (root/docs-only diff) it runs static checks, artifact bootstrap, the CLI prerequisite build, and boot smoke. Each step is bounded by the shared `runWithWatchdog` (class `changed`) so a hang fails fast, and it exits nonzero on the first failing step. This is purely additive: it does not change `pnpm test`, the merge gate, or CI, and the full suite stays available (`pnpm test:full`, non-blocking on push to main).

 `pnpm check:workspace-package-graph` verifies that every `workspace:` dependency or override in the root importer or a glob-matched workspace manifest resolves to a glob-covered workspace package, and that no package directory under `packages/` or `plugins/` falls outside `pnpm-workspace.yaml` package globs.

--- a/scripts/tests/verify-fast.test.mjs
+++ b/scripts/tests/verify-fast.test.mjs
@@ -10,9 +10,9 @@

 import test from "node:test";
 import assert from "node:assert/strict";
-import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
 import { tmpdir } from "node:os";
-import { dirname, join } from "node:path";
+import { dirname, join, resolve } from "node:path";
 import { fileURLToPath } from "node:url";

 import {
@@ -78,10 +78,48 @@ const PRETEST_CHECKS = [
  "scripts/check-workspace-package-graph.mjs",
  "scripts/check-no-test-timeout-appeasement.mjs",
  "scripts/check-changeset-format.mjs",
+  /*
+  FNXC:TestInfrastructure 2026-08-19-12:04:
+  Commit 027faaa09f added check-pre-json-anchor to the canonical pretest chain.
+  verify:fast was right while this hardcoded mirror was stale, so retain its
+  production position for the order-sensitive deepEqual drift guard.
+  */
+  "scripts/check-pre-json-anchor.mjs",
  "scripts/check-routes-modular.mjs",
  "scripts/check-runtime-skill-loader-drift.mjs",
 ];
 const STATIC_STEP_IDS = PRETEST_CHECKS.map((script) => `static-check:${script.slice("scripts/".length, -".mjs".length)}`);
+const PRETEST_VALIDATORS_START = "<!-- pretest-validators:start -->";
+const PRETEST_VALIDATORS_END = "<!-- pretest-validators:end -->";
+
+function extractDocumentedPretestValidators(markdown) {
+  const startCount = markdown.split(PRETEST_VALIDATORS_START).length - 1;
+  const endCount = markdown.split(PRETEST_VALIDATORS_END).length - 1;
+  if (startCount === 0) throw new Error("pretest validator block start anchor is missing");
+  if (endCount === 0) throw new Error("pretest validator block end anchor is missing");
+  if (startCount > 1) throw new Error("pretest validator block start anchor occurs more than once");
+  if (endCount > 1) throw new Error("pretest validator block end anchor occurs more than once");
+
+  const start = markdown.indexOf(PRETEST_VALIDATORS_START);
+  const end = markdown.indexOf(PRETEST_VALIDATORS_END);
+  if (end < start) throw new Error("pretest validator block end anchor precedes the start anchor");
+
+  return [...markdown.slice(start + PRETEST_VALIDATORS_START.length, end).matchAll(/^\s*-\s+`([^`]+)`\s*$/gm)].map(([, validator]) => validator);
+}
+
+function assertDocumentedPretestValidators(documented, canonical) {
+  const undocumented = canonical.filter((validator) => !documented.includes(validator));
+  const stale = documented.filter((validator) => !canonical.includes(validator));
+  const firstOrderDifference = Math.max(documented.length, canonical.length) === 0
+    ? -1
+    : Array.from({ length: Math.max(documented.length, canonical.length) }, (_, index) => index)
+      .find((index) => documented[index] !== canonical[index]) ?? -1;
+  assert.deepStrictEqual(
+    documented,
+    canonical,
+    `undocumented validators: ${undocumented.join(", ") || "none"}; documented-but-not-canonical entries: ${stale.join(", ") || "none"}; first order divergence: ${firstOrderDifference}`,
+  );
+}

 function stepIds(plan) {
  return plan.steps.map((s) => s.id);
@@ -164,6 +202,40 @@ test("buildVerifyPlan: defaults to every canonical pretest validator before esta
  }
 });

+/*
+FNXC:TestInfrastructure 2026-08-19-12:05:
+Pretest additions silently drifted both this hardcoded mirror and the testing guide.
+The guide's validator block is anchor-delimited and compared by exact production
+order; a substring scan was rejected because it cannot detect stale entries or reordering.
+*/
+test("docs/testing.md lists canonical pretest validators in exact order", () => {
+  const canonical = PRETEST_STATIC_CHECK_SCRIPTS.map((script) => script.slice("scripts/".length, -".mjs".length));
+  const documented = extractDocumentedPretestValidators(readFileSync(resolve(REPO_ROOT, "docs/testing.md"), "utf8"));
+  assertDocumentedPretestValidators(documented, canonical);
+});
+
+test("documented pretest validator guard rejects missing, stale, reordered, and unanchored blocks", () => {
+  const canonical = PRETEST_STATIC_CHECK_SCRIPTS.map((script) => script.slice("scripts/".length, -".mjs".length));
+  const block = (validators) => `${PRETEST_VALIDATORS_START}\n${validators.map((validator) => `- \`${validator}\``).join("\n")}\n${PRETEST_VALIDATORS_END}`;
+
+  assert.throws(
+    () => assertDocumentedPretestValidators(extractDocumentedPretestValidators(block(canonical.slice(1))), canonical),
+    new RegExp(`undocumented validators: ${canonical[0]}`),
+  );
+  const stale = "stale-validator";
+  assert.throws(
+    () => assertDocumentedPretestValidators(extractDocumentedPretestValidators(block([...canonical, stale])), canonical),
+    new RegExp(`documented-but-not-canonical entries: ${stale}`),
+  );
+  const reordered = [...canonical];
+  [reordered[0], reordered[1]] = [reordered[1], reordered[0]];
+  assert.throws(
+    () => assertDocumentedPretestValidators(extractDocumentedPretestValidators(block(reordered)), canonical),
+    /first order divergence: 0/,
+  );
+  assert.throws(() => extractDocumentedPretestValidators("# no validator block"), /start anchor is missing/);
+});
+
 test("buildVerifyPlan: no packages -> CLI prerequisite build then boot smoke", () => {
  const plan = buildVerifyPlan({ packages: [], staticCheckScripts: [], bootSmokeScriptPath: SMOKE, nodeBin: NODE });
  assert.deepEqual(stepIds(plan), ["bootstrap-artifacts", "build:@runfusion/fusion", "boot-smoke"]);
--- a/scripts/test-timings.json
+++ b/scripts/test-timings.json
@@ -1500,7 +1500,6 @@
        "packages/dashboard/app/components/__tests__/skills-view-mobile.test.tsx": 200,
        "packages/dashboard/app/components/__tests__/standing-instructions-template.test.ts": 100,
        "packages/dashboard/app/components/__tests__/step-number-alignment.test.tsx": 100,
-        "packages/dashboard/app/components/__tests__/stuck-and-agent-card-border.test.ts": 300,
        "packages/dashboard/app/components/__tests__/terminal-scrollback-floor.test.ts": 100,
        "packages/dashboard/app/components/__tests__/ui-metadata-sync.test.ts": 100,
        "packages/dashboard/app/components/__tests__/useGitHubStarCount.test.tsx": 100,
@@ -1785,7 +1784,6 @@
        "packages/dashboard/app/utils/__tests__/taskRevert.test.ts": 100,
        "packages/dashboard/app/utils/__tests__/taskStalledReview.test.ts": 100,
        "packages/dashboard/app/utils/__tests__/taskStatusBadgeLabel.test.ts": 100,
-        "packages/dashboard/app/utils/__tests__/taskStuck.test.ts": 100,
        "packages/dashboard/app/utils/__tests__/taskTiming.test.ts": 100,
        "packages/dashboard/app/utils/__tests__/taskTokenCost.test.ts": 100,
        "packages/dashboard/app/utils/__tests__/terminalPreferences.test.ts": 100,