Opt-in `pnpm smoke:pipeline` lane replaying 19 declared scenarios across
builtin:coding-ideas and the builtin:coding non-regression floor, driving the
real engine: disposable local Git repositories, throwaway PostgreSQL store,
production graph dispatch, ProjectEngine merge admission, real worktree
acquisition, and deterministic mock-provider scripts under testMode.
Each scenario declares one closed terminal state (merged-done, inert-intake,
parked, manual-hold, no-op-merge); an undeclared terminal fails the run, and
five wedge detectors (W1-W5) reject contradictory parks, finalization loops,
severed sessions, unreachable waits, and quiescence without progress.
Differential proof: on the pre-FN-180 tree (95ea06b48) exactly S05, S06, S09,
S10 and S16 fail across both workflows with behavioral assertions, and pass
after FN-180 — the FN-175/FN-177 incident classes are reproduced mechanically.
The declared duration budget is re-baselined 70s -> 90s at landing. The harness
did not degrade: the identical branch measured 61.8-64.1s against the
pre-integration main and 73.2-80.2s against the same main after 65 upstream
commits, with growth in transform, import and test phases the lane does not own.
docs/testing.md records the measurements, the cause, and the file-consolidation
lever to reach for before the budget is touched again.
Excluded from engine-default and engine-core; the merge gate is unchanged and
CI runs the lane non-blocking after merge.
Fusion-Task-Id: FN-182
139 KiB
Testing Guide
This guide consolidates the detailed testing guidance moved from AGENTS.md.
The merge gate
CI blocks PRs on exactly four checks (.github/workflows/pr-checks.yml): Lint, Typecheck, Build, Gate. The Gate job runs the boot smoke (scripts/boot-smoke.mjs: independent CLI --help and real fn init preflights run concurrently, the latter proving a durable .fusion/project.json marker, then a real fn serve answers GET /api/health, all against one isolated home) and pnpm test:gate: 15 static policy validators, 21 curated engine-core files, two PostgreSQL canaries, four core unit files, then the CI-shape test.
Set BOOT_SMOKE_TIMINGS=1 when invoking pnpm smoke:boot to print per-attempt help, init, health, and SIGTERM phase timings for diagnosis; the flag is off by default so normal gate output stays concise. Everything else — the 4-way shards, the engine slow tier, the dashboard inventory guard — runs NON-BLOCKING in .github/workflows/full-suite.yml on push to main.
Gate membership is the explicit allow-list in packages/engine/vitest.config.ts (engine-core project). Admission requires evidence of value (the test catches real regressions); tests never graduate in by default. A flaky gate test is evicted by deleting its allow-list line — the eviction PR does not need the flaky test to pass. The whole engine-core project must stay under ~60s wall-clock.
Static-validator and lane ordering: test:gate:static declares the 15 canonical, directly runnable read-only validators. scripts/run-static-gate-checks.mjs starts them concurrently and waits for every result, so zero, one, or multiple policy failures remain fail-closed and observable before tests start. It then starts engine-core, test:pg-gate, and test:unit-gate concurrently; the shell waits for all three and returns nonzero if any fail. CI-shape runs only after that successful wait.
FN-8783 warm result: The paired W32 protocol recorded in task document FN-8783/docs measured the complete-gate median at 15.4s baseline and 10.2s candidate across five serialized AB/BA pairs on the same macOS arm64 host (Node 26.3.0, pnpm 10.33.0, identical lockfile). The final engine-core transform-cache profile used one priming run (6.3s), then five warm runs (5.1, 5.2, 5.1, 5.0, 5.2s; median 5.1s) versus the pre-cache 6.2s focused engine-core result. The residual full-gate critical path is the unchanged concurrent engine/PG/unit/CI-shape work; task evidence records commands, SHAs, preparation, raw timing order, and coverage counts.
FN-9122 W33 re-measure: the reported 14.0s weekly row did not reproduce. After a build preflight and priming pass, three serialized warm pnpm test:gate samples measured 9.5s, 9.3s, and 9.9s (median 9.5s); post-ledger-repair samples measured 10.1s, 9.5s, and 9.0s (median 9.5s). A cold-cache sample was 10.9s. Static fan-out was 2.0s, engine-core 5.3s, PG gate 3.5s, unit gate 5.5s, and CI-shape 0.7s, so no gate-only seam explained a three-second increase. Treat an isolated weekly spike as a diagnosis trigger: retain raw samples and only change the gate after at least three warm samples have median above 11.0s. See the W33 gate investigation.
PostgreSQL and unit gate policy: packages/core's test:pg-gate intentionally runs handoff-to-review-atomicity.pg.test.ts and task-lifecycle-e2e.pg.test.ts, preserving atomic-handoff and lifecycle real-backend canaries. sync-workflow-ir-is-always-default.pg.test.ts was evicted under the merge-gate flake rule; its coverage remains in the non-blocking core suite (see the observed suite-only flakes register). test:unit-gate runs task-merge.test.ts, legacy-adoption.test.ts, no-hardcoded-lifecycle-columns.test.ts, and sync-workflow-ir-callsite-allowlist.test.ts. Every other former PG gate member remains enabled and discovered by the non-blocking command pnpm --filter @fusion/core test (default config: src/**/*.test.ts, no PG quarantine exclusions). scripts/__tests__/engine-vitest-gate-policy.test.mjs pins the exact two PG and four unit files, all waits, CI-shape ordering, and every engine/static member.
PostgreSQL fixture bootstrap: Do not hand-roll per-file CREATE DATABASE or drop helpers. Use createEmptyPgTestDatabase for migration-application and upgrade contracts, and createBaselinedPgTestDatabase only when an already-applied schema is the intended fixture state. Both keep database lifecycle and cleanup behavior aligned with the shared harness. Engine reliability fixtures follow the same bounded maintenance-connection and forced-cleanup contract; see the reliability helper DDL audit.
PostgreSQL connection-budget experiment
FN-9131 retains pg-connection-budget.ts as a tested cluster-shared advisory-lock primitive, but it is deliberately not wired into the shared harness. The experiment charges a backend slot at the pool ceiling (runtime + dedicated migration + admin), partitions fixed lease/work bands, derives a conservative degraded floor, and records bootstrap/admission counters (bootstrapRetries, bootstrapTokenWaits, bootstrapTokenReclaims, deburstDelayMs, floorAdmissionWaits, maxFloorWaitMs, lazyAdmissions, concurrencyRejections, and degradedCount). A non-zero degradation count invalidates a capacity measurement.
The initial harness wiring made a 27-worker PostgreSQL directory run worse, even after registry contention was changed from bounded rejection to queueing. Do not restore the wiring, alter test timeouts, add retries, or cap workers as a workaround. FN-9139 owns selecting a per-worker lifecycle point that is inert for non-PostgreSQL lanes and can charge admission wait outside individual test budgets. See the terminal-negative record.
Local PostgreSQL test server provisioning
Use pnpm pg:test:up, pnpm pg:test:status, and pnpm pg:test:down; forward flags as pnpm pg:test:up -- --replace and pnpm pg:test:down -- --purge. The script stages the pinned embedded PostgreSQL binaries under ~/.fusion/pg-test-server (or FUSION_PG_TEST_SERVER_HOME) and never starts a relative or worktree-native executable. dynamic_library_path cannot repair $libdir/plpgsql because PostgreSQL resolves that path from its executable.
The configured identity follows postgres@3.4.9: role is URL username, then PGUSERNAME, PGUSER, and the OS user; password is URL password then PGPASSWORD; database is URL path, then PGDATABASE, then the role name. A role-only repair therefore still fails when its role-named database is absent. The readiness gate runs configured login, PL/pgSQL, maintenance, and admin-DDL probes. A bare or CI-shaped probe is diagnostic-only unless its independently resolved role/database is one the script provisions; no harness connect dials a path-less base URL.
FUSION_PG_TEST_URL_BASE is shared by up, status, and down; export the same non-default URL for all three. A path-bearing URL is provisionable but reports harness-url-concat, since ${PG_TEST_URL_BASE}/${dbName} corrupts the 24 harness URL constructions. The script refuses URL/--port conflicts and PGHOST/PGPORT endpoint divergence. It reuses healthy servers, never mutates a foreign server's roles or databases, and requires a proven data-directory/PID ownership chain before --replace can stop a broken PostgreSQL server. pg_ctl daemonizes the postmaster, avoiding detached process spawning. A skipped pgDescribe block is not PostgreSQL verification evidence.
PostgreSQL setup-boundary participation and measurement
FUSION_PG_TEST_SETUP_PARTICIPANT=1 is the only opt-in for a PostgreSQL setup participant. It is intentionally accepted from a bare exported environment variable as well as the dedicated PostgreSQL configuration, so an ad-hoc directory reproduction has the same participation semantics. Any absent, malformed, empty, whitespace, or 0 value is non-participating; FUSION_PG_TEST_SKIP=1 overrides participation. Setup work on the shared file must leave non-participating lanes with no PostgreSQL import, socket, filesystem token, or environment mutation.
FUSION_PG_TEST_PREADMISSION_PROBE=1 is reserved for a default-off, observation-only measurement probe and is consulted only by a participating lane. A boundary probe must publish no token, lease, registry, singleton, or environment value that the harness can consume: coupling it would be an unmeasured admission rewiring, not a boundary experiment. FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS=1 enables the existing JSONL measurement channel in both arms.
Use scripts/pg-setup-boundary-probe.mjs to survey setup boundaries before changing a shared setup hook. Its temporary, connectionless fixture uses a falsifying four-arm budget matrix: only a duration-verified both-small pass proves off-budget work; a reference-only pass is completed-within-budget. Every cell is repeated, calibration controls must pass, and unclassified, timed-out, flapping, or indeterminate data is insufficient-data. The ledger compares monotonic clocks only within a process and requires an epoch separation margin across processes. See the recorded ownership survey. The campaign driver is executed only by a successor after that prerequisite is established; its pure unit suite is always safe to run. The loaded campaign is driven only by scripts/pg-preadmission-campaign.mjs, which alternates control and candidate arms, samples one shell's diagnostics, and enforces a five-minute run limit and sixty-minute campaign limit. Collect at least five valid runs per arm. The candidate is affordable only when its median wall time is no worse than the control maximum, its maximum failed-file count is no worse than control, its peak backend footprint is at most one above control, and it records no probe degradation. Fewer than five valid samples, invalid candidate probe records, a capped campaign, or an unavailable environment is deterministically insufficient-data, not a reason to tune timeouts or repeat unfavorable results.
Gate-safe @fusion/core barrel: the engine-core project resolves @fusion/core to packages/core/src/index.gate.ts (a project-scoped resolve.alias, not the root map), not the full packages/core/src/index.ts barrel. index.gate.ts is a byte-for-byte copy of the full barrel minus the export ... from statements for modules added to the barrel after the last re-audit baseline — i.e. it re-exports everything the full barrel does except genuinely new, gate-irrelevant feature modules (diffed against the prior baseline commit's barrel, not hand-picked from what gate test files import — production modules under test pull in far more of the barrel transitively than their own imports suggest). engine-default/engine-reliability/engine-slow are unaffected and keep resolving the full barrel. @fusion/engine is untouched (no gate file imports it). When adding a new barrel module that no gate test needs, mirror the exclusion in index.gate.ts rather than letting gate wall-time grow — see the FNXC comment at the top of index.gate.ts and packages/engine/vitest.config.ts's engine-core project for the audit procedure.
Pre-bundled @fusion/core gate bundle: FN-7668 profiled the gate's dominant wall-time cost as vitest/Vite SSR's import-phase — each fork worker independently re-resolves+evaluates the barrel closure from scratch with zero cross-fork sharing. engine-core's @fusion/core alias now points at a single esbuild-bundled ESM file (scripts/build-engine-core-gate-bundle.mjs, entrypoint packages/core/src/index.gate.ts, packages:"external" so only the first-party closure — 411 inputs in the FN-9122 re-measurement — is inlined) instead of directly at index.gate.ts's source, collapsing those per-fork Vite SSR module-loader round-trips into 1 file load per fork. The bundle is rebuilt fresh on every gate invocation via the engine-core project's globalSetup (the builder's own esbuild dependency graph determines what gets bundled — never a hand-maintained file/symbol list, so there is no drift surface), and lives at packages/core/.gate-bundle/core.mjs — a gitignored, non-committed artifact placed as a sibling of packages/core/node_modules/, deliberately not nested inside it: nesting inside node_modules triggers Vite's SSR external-dep heuristic (loads the whole bundle via Node's native loader, bypassing Vite's mock-interception pipeline) and silently defeats vi.mock for imports nested inside the bundle (see the FNXC comment in the builder script for the full repro/fix). Measured A/B (5 alternating runs each, FN-7669 task docs): median real wall-time −5.5%, import-phase aggregate −14.0%, transform-phase aggregate −25.9%, with full coverage parity (335/335 gate tests, identical per-file counts) — a modest but real, reproducible, zero-downside win. @fusion/engine stays on the full (unbundled) barrel: no gate file imports it directly, so bundling it would be zero-benefit churn against the core↔engine circular-import DI. Bundling the @fusion/engine relative-import graph (merger.ts et al., the untouched remainder of FN-7668's ~430-file closure) is a natural, larger-payoff follow-up, filed separately.
Workspace local/remote landing matrix
Workspace landing regressions must use real Git repositories under a non-Git workspace root. Cover a remote-free modified repository (local CAS and durable leases, with no remote Git operation), remote-enabled targets (configured/default remote fences and atomic push), and mixed workspaces where clean acquired peers have no review or landing obligation. The focused local-only regression is:
pnpm --filter @fusion/engine exec vitest run src/__tests__/workspace-e2e.test.ts --silent=passed-only --reporter=dot
Do not replace this with a structural store that omits workspace lease APIs: that shape can hide remote fence and intent behavior.
Weekly signal-per-second baseline
Refresh and publish the weekly test velocity baseline with the canonical process in Weekly test velocity baseline. That workflow measures the merge gate, boot smoke, and changed-only test lanes; appends scripts/test-velocity-history.json; and publishes docs/test-velocity-baseline.md. Keep the trend flat or net-negative; use its slowest-file and quarantine signals to drive FN-5048 rewrites and deletion-ratchet reviews instead of adding low-signal coverage. A single boot-smoke spike is not regression evidence: follow the FN-9105 controlled five-sample protocol, with BOOT_SMOKE_TIMINGS=1 and a median compared to 20,500 ms, before attributing or changing the boot path.
The gate's blind spot, stated honestly: typecheck + build + boot smoke + curated suite does not run the union suite a merge creates. Logic regressions outside the curated set land non-blocking by design — that is the accepted trade: the old broad gate caught no recalled real bugs while consuming ~70% of shipping time in flake triage.
Required workspace gates
Use the narrowest command that exercises the behavior you changed, then broaden before reporting completion.
pnpm test # gate suite + changed-only affected tests (bounded; never full-suite)
pnpm test:gate # the merge gate: curated engine-core suite + CI-shape test
pnpm smoke:boot # boot smoke: CLI --help + init marker + real serve /api/health
pnpm smoke:pipeline # opt-in, non-blocking real Git/PostgreSQL Coding pipeline smoke
pnpm verify:fast # TEST-FREE: static check:* gates + bootstrap + scoped typecheck/build + CLI build + boot smoke
pnpm test:full # full workspace suite — explicit opt-in only
pnpm lint # lint all packages
pnpm build # build workspace packages (excludes desktop/mobile; skips unchanged plugins safely)
pnpm verify:workspace # deep opt-in verification: lint -> test:full -> build (NOT the merge gate)
pnpm verify:fast (scripts/verify-fast.mjs) is the recommended test-free verification command. It first runs the canonical, read-only static validators from root pretest:
check-no-nohupcheck-no-cwd-relative-dashboard-test-readscheck-no-kill-4040check-no-getdatabasecheck-prerebase-inertcheck-capacity-pool-idcheck-cli-runtime-routingcheck-no-node-only-core-imports-in-dashboardcheck-pi-versions-pinnedcheck-workspace-package-graphcheck-no-test-timeout-appeasementcheck-changeset-formatcheck-pre-json-anchorcheck-routes-modularcheck-runtime-skill-loader-drift
check-runtime-skill-loader-drift enforces the Claude/Grok runtime skill loaders' clean rename-diff. It then bootstraps missing/stale workspace dist artifacts, runs typecheck + build scoped to the changed packages (reusing the same git-diff / changed-package resolution as pnpm test), always builds the @runfusion/fusion CLI package required by the source-checkout boot smoke, and runs the existing boot smoke once. The static phase invokes each existing validator entry point without update flags, is bounded and fail-fast, and runs no Vitest or test lane. It gives deterministic, flake-free signal in seconds, so it is a sound project testCommand/verification command when you want non-test verification. With no affected package (root/docs-only diff) it runs static checks, artifact bootstrap, the CLI prerequisite build, and boot smoke. Each step is bounded by the shared runWithWatchdog (class changed) so a hang fails fast, and it exits nonzero on the first failing step. This is purely additive: it does not change pnpm test, the merge gate, or CI, and the full suite stays available (pnpm test:full, non-blocking on push to main).
pnpm check:workspace-package-graph verifies that every workspace: dependency or override in the root importer or a glob-matched workspace manifest resolves to a glob-covered workspace package, and that no package directory under packages/ or plugins/ falls outside pnpm-workspace.yaml package globs.
Quality file-scoped preset
Commands emitted by the Quality file-scoped preset run the package-local binary as pnpm --filter <pkg> exec vitest run <package-relative paths>. Package ownership is resolved from the execution cwd, so task worktrees and disposable QA worktrees use their own workspace metadata. The emitted command passes no reporter flag and relies on Vitest's default reporter; Vitest 4 removed --reporter=basic, which must never be emitted. This applies only to the Quality command builder: configured --reporter=dot call sites and the documented executor scoped-verification command remain valid Vitest 4 usage.
pnpm build runs scripts/build-workspace.mjs: non-plugin workspace packages with build scripts still build on every run (excluding @fusion/desktop and @fusion/mobile), while plugin packages under plugins/ and plugins/examples/ can be skipped when .fusion/cache/plugin-build-cache.json records the same content hash as the current plugin package inputs plus declared local workspace-dependency inputs, root TypeScript/pnpm/build-tooling inputs, and all required dist/ outputs are present. A plugin rebuild is forced for a missing or partial dist/, no successful-build cache entry, changed tracked or untracked plugin/dependency/root build inputs, unavailable git content hash, or build-cache version changes. The cache is an optimization only; cache writes are best-effort and a failed package build still makes pnpm build exit nonzero with the planned package names.
pnpm test:full runs each package's default test script with capped worker fanout (FUSION_TEST_TOTAL_WORKERS=4 FUSION_TEST_CONCURRENCY=2 pnpm -r --workspace-concurrency=2 test). Do not casually raise worker counts; dashboard/jsdom and integration-heavy packages destabilize when oversubscribed. Use VITEST_MAX_WORKERS=<n> only for targeted package-level investigation.
Custom workflow reliability release signoff has a dedicated on-demand lane: pnpm test:workflow-release-check runs the manifest-listed targeted seams from scripts/lib/workflow-reliability-release-check.json, while --dry-run validates the manifest and prints planned commands and --json emits machine-readable item/seam evidence. This lane is not part of the merge gate and should not be added to test:gate or the engine-core allow-list.
Terminal acceptance tasks that require real mobile Safari should use docs/ios-acceptance.md for the --check run-vs-NO-OP probe, credential wiring, and physical/cloud real-iOS evidence workflow.
Agents running verification through fn_run_verification are bounded by default: project verificationCommandTimeoutMs when set, otherwise 300s for package scope and 900s for workspace scope, with an 1800s hard cap. Marathon invocations such as root pnpm test, pnpm test:full, pnpm verify:workspace, whole-package tests without file filters, and shell repeat loops are soft-capped unless the agent explicitly passes allowFullSuite: true; the escape hatch still emits progress heartbeats and respects the hard cap. Do not pass allowFullSuite: true unless absolutely necessary — it is the main way verification balloons past its budget. Default to a targeted, file-scoped command such as pnpm --filter @fusion/<pkg> exec vitest run src/path/to/test.ts --silent=passed-only --reporter=dot; reserve allowFullSuite for a genuinely full run with no targetable test set (state the reason), with the thin merge gate (pnpm test:gate) as the cross-cutting safety net.
Dashboard source-read fixtures
Dashboard app tests that inspect CSS or TypeScript source must use packages/dashboard/app/test/cssFixture.ts helpers such as readAppFile() and loadComponentCss(). Never read a bare relative path or construct a source path from process.cwd(); root-anchored Vitest launches otherwise fail at import time. scripts/check-no-cwd-relative-dashboard-test-reads.mjs enforces this convention in the full-suite pretest hook and merge gate.
Dashboard mutation-response clocks
Mocks that simulate updateTask or moveTask server responses must return a task with a strictly newer updatedAt. In TaskDetailModal suites, use makeUpdatedTask(current, patch) rather than rebuilding a response with makeTask({ ...current, ...patch }). mergeTaskSnapshot intentionally permits equal-clock sparse payloads to fill only absent fields; a frozen response can therefore retain populated detail metadata and falsely make a control appear not to repaint.
Fresh-worktree dist bootstrap
pnpm test auto-runs scripts/ensure-test-artifacts.mjs to rebuild missing/stale dist artifacts. Dashboard and dependency-graph package lanes auto-bootstrap too. If you hit opaque Failed to resolve import "./cli-spawn.js" (or similar), treat it as bootstrap regression against FN-4605 — don't work around with a manual pnpm build.
Public @fusion/core exports consumed by runtime tools should include a literal built-dist guard (for example importing packages/core/dist/index.js) when package test aliases otherwise resolve @fusion/core to source.
Engine static process guards
packages/engine/src/__tests__/user-configured-command-no-execsync.test.ts guards user-configured command execution helpers against accidental execSync usage or dropped async bounds. Its registry covers verification helpers, fn_run_verification, executor configured-command execution, merger post-merge script execution, routine command execution, and the native/bubblewrap/sandbox-exec sandbox backends. Each protected slice must keep the appropriate bounded async safeguard (timeout/timeoutMs, maxBuffer, or maxLifetimeMs). The test intentionally slices named function bodies instead of scanning whole files; deterministic git-plumbing execSync in merger/self-healing/already-merged/integration/worktree-prune paths and the executor git ancestry check are explicitly out of scope.
FNXC future-date advisory
pnpm check:fnxc-future-dates continues to block new future-dated stamps while reporting a non-blocking advisory for the existing population. It scans only packages/, scripts/, and docs/, classifying each future stamp as timezone-plausible (≤26h ahead), suspect (>26h through 48h), or implausible (>48h). Run pnpm check:fnxc-future-dates -- --report-anomalies for a read-only full census grouped by file and area; report mode never writes the baseline or fails the build.
Lifecycle-column census (report-only)
pnpm census:lifecycle-columns reports every comparison against the six legacy column ids
(triage, todo, in-progress, in-review, done, archived) across the packages and plugins
source trees, with comments stripped. It reports four separate numbers, and that separation is
the whole value — three of the four must NOT be converted, and every one of them was silently
inside the single tracked figure:
- COLUMN guards — the real backlog. A lifecycle decision made by column NAME stops matching the moment a board renames a column.
- ROLE comparisons —
role === "triage",agentType === "triage",entry.agent === "triage". These compare an AGENT ROLE. The planner lane is namedtriageand keeps that name; U11 removed only the column. These must NOT be converted — renaming the role silently empties the planner's prompt template and mis-binds its model markers. - STATUS comparisons —
step.status === "done",goal.status === "archived",feature.status === "done".StepStatusispending | in-progress | done | skipped, and missions, goals and features carry their own statuses; three of those names collide with column ids. This is the largest correction the census makes — 182 sites, inflatingdoneby 105 andin-progressby 49. Converting one is a category error: asking which column carries thecompletetrait about a STEP's status would stop the step reading as finished. - DELIBERATE-LITERAL — reviewed sites whose literal is correct, with the reason recorded at the
site rather than in a list that can drift from it. Grep
DELIBERATE-LITERALto enumerate them.
Why it exists: the program tracked its remaining work by grepping === "triage", and that count
was simultaneously too low (six ids exist; triage was under 4% of the total, and the pattern was
anchored on locals named column, so real guards on from and originColumn were invisible) and
too high (12 role comparisons and 182 entity-status comparisons counted as backlog). A count that is
wrong in both directions sends work to the wrong files and hides the files that need it.
The two non-column classes are recognised structurally, not by a name list, because names are
unbounded and a name list was already wrong twice (sessionPurpose, surface). AgentRole is
triage | executor | reviewer | merger and StepStatus is pending | in-progress | done | skipped;
the members that are NEVER column ids (executor/reviewer/merger, pending/skipped) identify
which vocabulary an expression belongs to whatever its variable is called.
The classifier is AST-based (scripts/lib/lifecycle-column-census-ast.mjs, ts.createSourceFile),
because three people measured this backlog with three greps and got three different answers for the
role bucket (6, 8, 12). A regex cannot tell a column guard from an agent role, a session purpose, a
surface name, a step status, or a comment. The parser also sees shapes no per-line pattern can:
multi-line comparisons, literal-on-the-left, loose equality, and JSX. The text classifier is kept
beside it as an independent second implementation — --compare asserts the parser is a strict
SUPERSET of the regex (measured +6, all real) and FAILS if the regex ever finds something the parser
misses, which would mean the parser has a blind spot and its count cannot be the bar.
What the parser still cannot do, stated rather than implied: without a full type-checker program it cannot prove a receiver is column-typed, so classification remains evidence-based (receiver name plus the vocabulary its siblings use). That is why the four classes are reported separately and never netted — a wrong classification stays visible instead of silently moving the bar.
--json emits the machine-readable form. --strict compares per-file counts against
scripts/lib/lifecycle-column-census-baseline.json:
- a rise fails hard — that is the ratchet's purpose, "no new guards";
- a drop reports that the baseline can be tightened and exits 0 without writing it.
A dropped baseline must be deliberately re-recorded and committed with
--strict --update-baseline. --strict --exact restores hard failure on a drop, for the end state where the
count is pinned and any divergence is a real event. --strict --update-baseline re-records unconditionally
and prints ACCEPTED RISES, which is the only way to record a rise deliberately.
The regression suite is packages/engine/src/__tests__/lifecycle-column-census.test.ts. It pins
each form the census must catch (all six ids, non-column locals, single quotes, negation,
multiple hits per line) and each it must not (role comparisons, comment prose, trailing line
comments, marked sites) — plus that one marker cannot launder a distant guard in the same file.
The CLI additionally exits non-zero on an empty file list, because a guard that reports success
without checking anything is worse than no guard.
Dashboard Availability & Supervised Mode
When running the dashboard for extended UX review sessions (e.g., Atlas Notes Jony pass via Tailscale Serve), use the --supervise flag to prevent unexpected dashboard exits from leaving the Tailscale endpoint returning 502:
fn dashboard --project atlas-notes --port 4040 --supervise
The supervisor runs the dashboard as a child process with bounded restart attempts (one initial run plus up to 3 restarts with exponential backoff: 2s → 4s → 8s). Clean exits (SIGINT, SIGTERM, exit 0) propagate without restart. If the child crashes repeatedly, the supervisor gives up after the retry budget and prints actionable diagnostics including the actual restart command and health-check curl.
Key invariants:
- Planning sessions that receive non-JSON AI output persist as retryable error state (not process exit)
/api/healthremains available during planning session errors- Remote Tailscale 502 means the local listener on
127.0.0.1:4040is absent — restart the dashboard - Check local health:
curl http://127.0.0.1:4040/api/health
Process management guardrails still apply: The supervisor does NOT use nohup, shell kill loops, or unbounded retries. It never kills existing processes on port 4040.
Dashboard Test Lanes
pnpm --filter @fusion/dashboard test # curated app/API quality gate (default)
pnpm --filter @fusion/dashboard test:deep # exhaustive app + API suite
pnpm --filter @fusion/dashboard test:app # exhaustive React/jsdom
pnpm --filter @fusion/dashboard test:api # exhaustive Node API/server
pnpm --filter @fusion/dashboard test:browser-smoke # local browser CSS/layout smoke
pnpm --filter @fusion/dashboard test:build # built client output contract
Run test:deep when changing broad dashboard architecture, shared modal/view infrastructure, or route registration. Run test:browser-smoke for layout/responsive/navigation/modal/CSS changes. Run test:build for Vite output, lazy-loading, chunking, or client-dist changes.
The dashboard CSS contract lane includes app/__tests__/dashboard-css-token-validity.css.test.ts, which scans raw component/app CSS and fails any var(--token) reference that is not defined by CSS, assigned by React inline style, or explicitly allowlisted as runtime-local. Run it with component-css-no-raw-rgba, dashboard-component-color-tokenization, and text-token-canonicalization when touching design-token usage.
Command Center responsive chart fixes need evidence beyond jsdom. Keep the jsdom scroll-owner tests for rule/structure coverage, but pair them with packages/dashboard/app/components/command-center/__tests__/CommandCenter.mobile-chart-layout.test.ts, which reads the co-located Command Center CSS files directly and asserts the mobile shrink/height/border rules that real layout depends on. For visible defects, also capture a real browser/device (or headless Chrome/Blink) reproduction with scrollWidth > clientWidth, zero/clipped clientHeight, or stretch measurements; do not close a Command Center mobile chart bug on jsdom-green assertions alone. The local pnpm --filter @fusion/dashboard test:browser-smoke --require-browser lane now includes [data-smoke="command-center-charts"] and gates representative Command Center recharts pie, line, and empty states at 390×844 mobile plus desktop viewports for visible SVG/container height, overflow containment, empty-state text, and chart scroll-owner violations.
The same required-browser lane also measures the Agents Overview fixture at 390×844 mobile and 1280×700 short-desktop viewports. It verifies the real Active Agents scroll owner overflows, reaches the final card after scrolling, preserves sibling Agents content, avoids horizontal page overflow, and leaves the metrics-only empty state unclipped.
The Quick Add Save fixtures at the supported 300px minimum derive both their locale set and expected count from SUPPORTED_LOCALES plus each shipped tasks.save catalog entry; adding a locale needs no smoke-script edit. The fixture HTML escapes catalog labels to preserve React-equivalent text rendering, and browser-layout-smoke-fixture.test.ts uses an injection seam to guard locale derivation drift, missing translations, and escaping in the fast jsdom lane.
The shared mobile/tablet overflow-containment net lives at packages/dashboard/app/__tests__/dashboard-overflow-containment.test.tsx. It covers board/kanban columns, task-detail modal shell, workflow/simple workflow editors, and Activity Log modal at mobile, tablet, and landscape-phone breakpoints. Run it directly when touching dashboard viewport containment or shared modal/workflow CSS:
pnpm --filter @fusion/dashboard exec vitest run --project dashboard-app app/__tests__/dashboard-overflow-containment.test.tsx --silent=passed-only --reporter=dot --exclude '**/build-output.test.ts'
pnpm --filter @fusion/dashboard test runs the curated app/API quality gate through
packages/dashboard/scripts/run-quality-tests.mjs (FN-6308). The orchestrator keeps
the historical app/API quality split and the curated/backfill lane boundaries, but
schedules independent lanes with bounded process concurrency instead of chaining every
Vitest launch sequentially. Each lane still runs through
packages/dashboard/scripts/run-vitest-with-heap.mjs --heap=6144; do not bypass that
wrapper or recombine the jsdom-heavy app/API projects, because the old combined run
was SIGKILLed by heap pressure under workspace worker budgeting. The top-level
pretest artifact bootstrap runs once before the orchestrator; lane subprocesses must
not re-run scripts/ensure-test-artifacts.mjs.
Use the exact aggregate command below to attempt all 15 app/API lanes, including after one or more lanes fail:
pnpm --filter @fusion/dashboard test -- --all
pnpm forwards that invocation as -- --all; the quality runner deliberately removes
only the leading package-script separator and still rejects genuine unknown options.
--no-fail-fast is an equivalent aggregate alias. Without either alias, the default is
fail-fast for quick local feedback: remaining lanes are reported as NOT RUN with
UNKNOWN status, never as passing. Aggregate mode attempts every lane exactly once;
any failed or signal-terminated lane keeps the command nonzero and is named in the
final failure summary. It must never print an all-passed summary when any lane failed.
When scripts/test-changed.mjs runs affected-package vitest --changed scopes, @fusion/engine and @fusion/dashboard are each split out from other scopable packages into their own dedicated memory-envelope run: NODE_OPTIONS=--max-old-space-size=6144 plus FUSION_TEST_TOTAL_WORKERS=1, FUSION_TEST_CONCURRENCY=1, and VITEST_MAX_WORKERS=1. All other scopable packages remain in the shared non-envelope group, and packages without a Vitest config still fall back to their package test scripts. The envelopes use a scoped affected watchdog ceiling below the default workspace verification timeout, so the expected failure mode is a normal Vitest pass/fail or script watchdog timeout, not raw pnpm SIGKILL or executor timeout restart. Re-measure with a wide changed selection (for example a dirty packages/core/src/index.ts boundary edit for engine, or an App/jsdom-affecting dashboard diff) before changing either envelope.
For the heavy envelope packages this lane is additionally bounded against wide vitest --changed graph fan-out: when a changed non-test source file falls in the package's own dir or any transitive workspace-dependency dir, the affected lane runs only the directly-changed test file(s) and delegates the rest to the merge-gate suite, so a single hub edit can no longer expand into a near-full suite that exceeds the 15-min verification timeout. Pure test-file changes still run the normal vitest --changed scope.
Concurrency knobs:
FUSION_DASHBOARD_TEST_CONCURRENCYcontrols dashboard quality lane process concurrency, defaulting to2and hard-capped at2to preserve the measured heap budget.- Per-lane heap is fixed at
6144MiB by the orchestrator. Treat any code change that makes this configurable or increases it as risky and re-measure for OOM/SIGKILL before landing. FUSION_TEST_TOTAL_WORKERS/FUSION_TEST_CONCURRENCY(or targetedVITEST_MAX_WORKERS) still bound Vitest thread fan-out inside each process viacomputeMaxWorkers; do not raise them casually for dashboard/jsdom runs.
New test files under app/** or src/** are picked up automatically by the
backfill lanes (dashboard-app-quality-backfill / dashboard-api-quality-backfill),
which include the broad globs and exclude only the files an explicit curated lane
already runs plus the skip-list. You do not need to register a new file by hand for
it to run — the curated-gate hole that silently skipped unenumerated files is closed
(see "Curated-gate completeness" below). Add a file to a curated qualityApp*/qualityApi
list only when you want it in a specific fast lane rather than the backfill catch-all.
When a dashboard element is "missing", probe before theorising
getByText / getByRole failures name the element that was not found, which is almost never where
the fault is. Three separate causes in App.test.tsx and board-mobile-view-switch.test.tsx all
presented identically as a missing element, and in each case the DOM looked healthy:
| what the test said | what was actually wrong |
|---|---|
Unable to find "+ New Task" |
ListView rendered its workflow SKELETON, which carries the same list-view class as the real body, so the preceding waitFor(".list-view") passed |
Unable to find role="heading" "New Task" |
NewTaskModal THREW — an incomplete vi.mock was missing isShortViewport — and an ErrorBoundary swallowed it |
Unable to find [data-testid="switch-to-board"] |
an uncaught render error unmounted the entire React root; the DOM was empty three lines earlier |
Three theories were offered for these before any probe, and all three were wrong ("the board renders nothing", "i18n is returning keys", "the FloatingWindow rework"). Three probes each landed the cause on the first try. The technique is simply to print what is really there at the failing assertion:
// eslint-disable-next-line no-console
console.log(
"PROBE testids:", JSON.stringify([...document.querySelectorAll("[data-testid]")]
.map((e) => e.getAttribute("data-testid")).slice(0, 12)),
"| boundary:", document.querySelector('[class*="error-boundary"]')?.textContent?.trim() ?? "none",
);
What each signal means:
- an
error-boundaryclass in the DOM — a component threw and the boundary ate it. Read its text; for a mock-related throw it names the missing export exactly. - an empty DOM with no boundary — an uncaught render error unmounted the root. Everything after it fails misleadingly, so trust the FIRST failing assertion, not the reported one.
- the element's container present but its contents absent — a skeleton or empty state is
standing in. Check whether the selector you waited on is shared with that state;
list-viewis, which is whylist-view-bodyexists.
Corollary for writing assertions: wait on a marker only the real thing has. A class shared with a loading or empty state turns "the list rendered" into "something rendered", and the test then fails one line later against a DOM that looks fine.
Curated-gate completeness and the skip-list
The dashboard quality gate is a chain of curated lanes plus two backfill lanes.
Together they must execute every *.test.{ts,tsx} under packages/dashboard/app
and packages/dashboard/src, or the file must be on the reviewed skip-list. This is
enforced by a guard (CI job Dashboard curated-gate guard in full-suite.yml, non-blocking):
node scripts/check-test-inventory.mjs --dashboard-curated
It fails when a dashboard test file is neither executed by a quality project nor
skip-listed. The skip-list lives at scripts/lib/dashboard-curated-skiplist.json;
every entry needs a non-empty reason (empty reasons are rejected). Skip-list policy:
- A file goes on the skip-list only when it genuinely cannot be gated yet — today
that is pre-existing-failing orphans (tests that were never executed in CI and
fail in isolation) and
build-output.test.ts(runs standalone viatest:buildafter a Vite build). Each carries a one-line reason. - Every skip-list `reason` for a pre-existing failing/orphaned test must reference a concrete `FN-NNNN` tracking task; if the test is rescued, remove the entry instead of leaving a tracking placeholder.
- The guard rejects any skip-list entry whose file is already executed by a quality project. Remove the entry instead; the skip-list is only for genuinely non-executed files.
- To remove a file from the skip-list: fix the test, confirm it passes under its project, delete the skip-list entry. The backfill lane then executes it.
- The skip-list is shared verbatim with
vitest.config.ts, which excludes the same globs from the backfill projects — one source of truth.
Test-inventory harness
Engine test-harness integrity also includes
packages/engine/src/__tests__/vi-mock-specifiers-resolve.test.ts.
Engine relative-specifier guard
The guard resolves literal relative specifiers in vi.mock, vi.doMock,
vi.unmock, vi.importActual, and vi.importMock; typeof import("…") type
positions (including importOriginal<typeof import("…")>()); and static or dynamic
imports in engine test files. It intentionally ignores package aliases, node:
specifiers, non-literal dynamic imports, and quoted/template fixture samples.
The engine tsconfig excludes its tests directory, so typecheck is not a substitute
for this check. When a folder refactor moves a module, update the vi.mock target
and every importActual, unmock, typeof import(...), and ordinary-import
sibling in the same change. The guard ratchets explicitly allowlisted historical
dead vi.mock targets downward; do not add newly discovered stale paths to that
allowlist.
scripts/check-test-inventory.mjs is the standard coverage-superset verification
step. Node stdlib only.
# Snapshot the executed-test inventory (per package/project, normalized test ids).
node scripts/check-test-inventory.mjs --capture before.json
# ... make a change ...
node scripts/check-test-inventory.mjs --capture after.json
# Fail (exit 1) if any test id present in `before` is missing from `after`.
node scripts/check-test-inventory.mjs --diff before.json after.json
The capture spec (which packages/projects to enumerate) lives in
scripts/lib/test-inventory-spec.json. The diff lists the exact missing test ids;
a renamed file shows up as a remove (old path) + add (new path), so the rename is
reviewable. New test ids never fail the diff.
Pipeline smoke lane (opt-in, non-blocking)
pnpm smoke:pipeline runs the engine-pipeline-smoke Vitest project through
scripts/run-pipeline-smoke.mjs. It is the AI-free, network-free regression net for
the FN-175/FN-177 class: merge admission before a Code Review verdict, a failed card
whose branch has already landed, and cleanup that removes a live executor worktree.
It drives disposable local Git repositories, a throwaway PostgreSQL store, the real
built-in Coding (Ideas) and Coding workflow definitions, real merger admission, and
deterministic mock-provider scripts under testMode: true.
Prerequisites are Git and reachable test PostgreSQL. Start the latter with
pnpm pg:test:up; the wrapper fails with that actionable instruction when either
prerequisite is absent (or accepts --allow-skip only for an explicit local
non-execution). The workflow project is intentionally excluded from engine-default
and engine-core so it cannot expand pnpm test or the merge gate. The non-blocking
Pipeline smoke tier job in .github/workflows/full-suite.yml runs after merge with
fetch-depth: 0 and a PostgreSQL service; never add it to pr-checks.yml, branch
protection, or the engine-core allow-list.
The declared budget is 90 seconds, rounded up from the slowest warm PostgreSQL run
measured against current main (73,224ms, 73,866ms, 76,480ms, and 80,162ms) after the
integration of upstream engine growth. The wrapper enforces it for every run; an overrun is a result
to investigate, never a reason to hide a regression behind unbounded timeouts. Use
--repeat=10 for the reproducibility proof, --json for machine output, and
--budget-ms=<n> only for loud diagnostic measurement. The normalized report lists
scenario, variant, workflow, expected terminal, observed terminal, verdict, and duration.
Each scenario declares one closed terminal state: merged-done, inert-intake,
parked, manual-hold, or no-op-merge. The harness fails on an undeclared terminal
(including an unexpected merge), and treats these as unconditional wedges: W1
contradictory park, W2 repeated finalization/work-item loop, W3 severed live
session, W4 unreachable held/runnable work item, and W5 bounded quiescence
without progress. A parked or manual-hold scenario must also prove its declared
recovery reaches merged-done.
Adding a pipeline scenario
- Add exactly one
SNNentry toPIPELINE_SCENARIOSwith its terminal state, trait-resolved workflow coverage, executablearrange/actdrivers (and recovery when needed), and a hard persisted-state/Git invariant. - Implement those drivers in
_pipeline-drivers.tsthroughPipelineSmokeHarness; keep the nominal/review, concurrency/merge, and resilience test files as scenario-to-report adapters. - If its terminal is
parkedormanual-hold, implement and assert a recovery tomerged-done; never convert it into an implicit success. - Extend
CODING_NON_REGRESSION_FLOORwhen the scenario also applies tobuiltin:coding; document any Ideas-only exemption in the table. - Run
pnpm smoke:pipelineand preserve the measured report. Do not add the file to default, slow, or gate projects.
Engine slow tier (non-blocking CI)
The engine-slow vitest project (packages/engine/src/**/*.slow.test.ts) holds the
long real-git suites. It runs locally via pnpm --filter @fusion/engine test:slow and
in CI via the Engine slow tier job in full-suite.yml (non-blocking, push to main), which uses
scripts/assert-engine-slow-nonempty.mjs to fail if zero tests executed (so a glob
or config drift that silently empties the tier breaks the run instead of passing vacuously).
The CI job uses fetch-depth: 0 because these tests run real git operations.
Quarantine ledger and the deletion ratchet
Flaky tests are quarantined ON SIGHT and deleted on a 2-week clock. The ledger remains a simple JSON record, while scripts/check-quarantine-ledger.mjs mechanically checks every ledger/exclude pair (see the AGENTS.md standing rule "Flaky Tests Are Quarantined on Sight").
Quarantine is the default when a sighting is reproducible enough to justify evicting a file's coverage. The only exception is the narrow first-sighting record authority in AGENTS.md: a high-value file may be recorded in the observed suite-only flakes register instead (docs/solutions/test-failures/suite-only-flakes-observed-register.md). A second sighting of a registered flake moves it to the ledger plus matching Vitest exclude in one lockstep commit.
Merge-gate eviction is a separate branch. A flake inside the blocking merge gate is evicted from its allow-list or canary script, not skipped and not timeout-widened; the eviction does not need the flaky test to pass. Coverage may remain in the non-blocking suite. Quarantining a PostgreSQL file additionally conflicts with the gate-policy assertion that quarantinedCoreTests stays empty, so that owner decision is escalated rather than performed inline. FN-8928's default-IR canary record is the worked example: FN-8912 observed a loaded-lane setup-hook timeout, so the file was evicted rather than quarantined.
To quarantine a test (a test that failed without a corresponding real bug in the change), in one commit:
- Add an entry to
scripts/lib/test-quarantine.json:{ "file": "<repo-relative test path>", "reason": "<why + link to the failing run>", "quarantinedAt": "YYYY-MM-DD" } - Add a matching one-line
excludeentry to that package's vitest config.
The clock: an entry expires 14 days after quarantinedAt. Whoever touches the suite and finds an expired entry deletes the test file, its ledger entry, and its config exclude (git history is the archive). scripts/check-test-inventory.mjs --diff stays deliberately unwired in CI because it would fail on exactly these deletions.
Quarantine deadline visibility check
Run pnpm check:quarantine-ledger to print a soonest-deadline-first summary of scripts/lib/test-quarantine.json. The command uses the same 14-day deletion clock (quarantinedAt + 14d) as the velocity baseline and reports each entry as expired, near-deadline, healthy, or unknown when quarantinedAt is missing/invalid. Default mode remains report-only for deadline status.
The checker also enforces the quarantine lockstep. It reads only comment-stripped exclude: array literals in every packages/*/vitest.config.ts; include-shard lists and identifier/spread excludes are deliberately out of scope. It reports missing-file (a ledger entry names no file), missing-exclude (a ledger file lacks its package exclusion), and dangling-exclude (an exclusion names no file). --strict fails on any of those violations as well as near/expired deadlines.
Flags:
--warn-within=<days>changes the near-deadline window from the default 5 days.--jsonemits the computed rows, lockstep violations, and summary counts for machine consumption.--strictexits 1 when an entry is expired/near-deadline or a lockstep violation exists; this is enforced by the PR check.
Rescue (before the clock runs out) requires both: evidence the test catches real regressions, and a root-cause fix for the flake. Stabilization passes — widened timeouts, retries, loosened assertions — are appeasement, not rescue, and are banned (for agents especially).
2026-08-09 Kimi K3 disposition (FN-8900): Rescued register-model-routes-kimi-k3-supplemental.test.ts before the 2026-08-15 deadline. The route test now consumes pi-ai 0.82.1's real bundled K3 catalog via a deterministic registry seam, retaining SDK-catalog regression signal and route merge/deduplication coverage without ModelRuntime.create() or live ModelRegistry.refresh(). The re-measurement recorded refresh samples of 224, 74, 75, 279, and 300047 ms (the last process took 300.50 s), so the ledger entry and dashboard Vitest exclusion were removed together rather than widening the unchanged 15 s budget or adding a retry.
2026-08-10 Planning Mode disposition (FN-8936): Rescued PlanningModeModal.planning-flow.test.tsx before its 2026-08-20 deadline. Investigation found no product state-machine race: Proceed snapshots stable session/summary refs and takes its single-flight guard before create. The loaded failure was a test harness detached-node race when session hydration replaced an action-bar button returned by findByRole. All unsafe direct Proceed handoffs now settle hydration and query a live button before clicking, while retaining strict create arguments, task-created/onTaskCreated, desktop/mobile handoff, claim-retry, retry, and multi-task assertions. Exact and loaded-file runs passed, and the matching ledger entry and dashboard Vitest exclude were removed together without changing timeouts or adding retries.
2026-08-10 project-engine disposition (FN-8937): Rescued project-engine.test.ts before its 2026-08-20 deadline. The suite's un-mocked exec-based integration-branch probe spawned real git, while the shared subprocess guard watchdog used fakeable timers; a duplicate-registration path could also orphan a watchdog handle. The resolver is now a deterministic suite seam, and watchdogs use captured real timers with owner-scoped failure draining, preserving sibling-test failure ownership. The ledger claim that runtime schedules 120s was a misread of FUSION_TEST_SUBPROCESS_TIMEOUT_MS: production retains its correct 60s ladder at packages/engine/src/project-engine.ts:4551, and the test correctly forbids its uncapped 120000ms rung. Thus the request to update the assertion to 120s is a documented deviation; no timeout was widened, retry added, or assertion weakened. The ledger/config exclusions were removed together, while the file remains outside engine-core pending separate gate-admission evidence.
2026-08-17 plugin-runner disposition (FN-9141): Rescued packages/engine/src/__tests__/plugin-runner.test.ts before the 2026-08-30 deletion-ratchet deadline. The seed-recorded completed lane combined shuffled order, no-isolation worker reuse, and a temporary byte-for-byte subject repeat. It reproduced one hot-reload warning assertion failure in the original subject because an unrelated worker-reused file cleared createLogger.mock.results; the repeat passed. The suite now keeps the initialized mock logger in a stable hoisted reference and explicitly proves cleanup cannot erase it, retaining the strict stopPlugin rejection/warning and renamed-complete-lane onTaskCompleted dispatch contracts. The ledger row and default-lane exclusion remain removed together. No timeout, retry, assertion weakening, skip, polling, or permanent worker-policy change was used.
Validate before excluding and preserve timeout budgets
Capture full runner output before recording or filing a ledger entry—for example, tee it to a file. Never pipe a dot reporter through tail: the summary remains but the FAIL identity lines needed for evidence are truncated.
Validate a quarantine-bound file before adding its exclusion. The dashboard quarantine array is spread into every dashboard project exclude, so even an explicitly named CLI file is suppressed afterward; no CLI flag removes a configured exclusion. The only local route back to validation is an uncommitted removal of both lockstep entries. Hoisting expensive real-dependency construction into a reusable per-file beforeAll is a valid structural rescue, but it inherits the hook timeout and does not by itself fix a duration-driven flake. Do not widen a timeout under a “deliberate budget” framing without an owner-approved policy exception; FN-8647 and #3245 document this distinction.
When proving that a quarantine change did not alter the budget, inspect the staged diff before the final lockstep commit and fail nonzero on any added or removed config testTimeout, hookTimeout, or teardownTimeout line—removal falls back to a runner default. Then parse the resulting test source rather than applying a line-wise diff regex: any expression in a hook's second or case's third timeout position is forbidden regardless of its shape (15 * 1000, a bare identifier, or a cast all count). Resolve calls through a vitest import alias map and namespace bindings; reject local rebinding and computed access outright. Rebinding detection must scan the whole initializer/assignment RHS, not one root identifier, so container forms such as const [h] = [beforeAll], object/conditional/sequence wrappers, and element access are caught while direct invocation callee positions are skipped.
Inspect options objects recursively through inline object-literal spreads. Reject every non-inline spread and every computed option key; identifier-spread immutability/dataflow proofs are unsound under direct and alias mutation. An out-of-tree guard must resolve TypeScript from the repository CWD with createRequire(path.join(process.cwd(), "package.json")); an unrunnable guard is a hard failure. For a non-collection check, classify Vitest's status and no-files diagnostic first, then parse only the JSON test-file list: the diagnostic itself echoes the requested path.
Vitest timeout-appeasement guard
scripts/check-no-test-timeout-appeasement.mjs runs in the fast pretest, pretest:full, and test:gate paths. It scans tracked packages/**/*.test.* and plugins/**/*.test.* files for per-file or suite-level Vitest timeout bumps, including vi.setConfig({ testTimeout: ... }), vi.setConfig({ hookTimeout: ... }), and bare testTimeout: / hookTimeout: properties in test files. It deliberately ignores global vitest.config.* timeouts.
Legitimate legacy exceptions must be recorded in scripts/lib/test-timeout-appeasement-allowlist.json as { "file": "<repo-relative test path>", "reason": "<owning cleanup/quarantine task and rationale>", "allowlistedAt": "YYYY-MM-DD" }. Allowlisting is temporary: the real fix is to quarantine the flaky test or narrow the slow seam, then remove both the timeout bump and the allowlist entry.
CLI shared-fixture rescue pattern (FN-6430): the 2026-06-14 @runfusion/fusion quarantine batch passed direct runs but timed out or bled state only under package/workspace load. The rescue fixed the shared isolation seam, not the timeout: sweep stale top-level fn-test-home-* roots with a bounded one-level prefix scan, reject inherited HOME values that do not live under the current fusion-test-workers-* root, recreate/remark the worker root before each mkdtemp, reset module/singleton fixture state in the affected suites, close real stores created by research helpers, and narrow slow real-store seams by moving package imports out of timed test bodies. When rescuing a similar CLI batch, prove it with repeated rescued-file runs plus pnpm --filter @runfusion/fusion test, audit rescued files for vi.setConfig/testTimeout/hookTimeout appeasement, and keep ledger/config removals in the same commit.
Non-CLI quarantine sweep pattern (FN-6433): for engine/core/dashboard batches, first remove quarantine excludes only in temporary local configs and run the exact quarantined files together so suite-load coupling is visible before editing the ledger. Rescue is valid when the grouped package lane proves the invariant now holds (for example, FN-6433 fixed engine cross-file interference by replacing broad activeSessionRegistry.clear() cleanup with path-scoped unregistering) or when a prior shared-fixture fix is demonstrated under package load. Delete duplicate/low-value files under the ratchet when another deterministic suite owns the same invariant. Finish by making scripts/lib/test-quarantine.json and every package Vitest exclude array converge in one commit, then prove the empty/non-empty state with package lanes, pnpm test:gate, pnpm test, pnpm build, and the bounded temp-leak output from pnpm test.
2026-06-15 rescue batch (FN-6486): two same-day quarantines were rescued before their 2026-06-29 deletion deadline. store-concurrent-writes.test.ts kept its WAL/transactionImmediate regression value by making the external lock helper's timed release use synchronous Atomics.wait inside the child process, removing event-loop timer scheduling as the load-only flake source without widening retry windows. extension-task-tools.test.ts kept its worktree-root task-tool coverage by closing each real TaskStore fixture before temp-root removal and using non-hoisted mock cleanup. The reusable pattern is to remove scheduler/resource leaks in the helper or fixture seam, then prove the rescue with repeated exact-file runs plus package lanes, not with timeout bumps, retries, assertion loosening, or worker changes.
2026-06-17 core cleanup rescue (FN-6600): a broad @fusion/core timeout cluster was accompanied by fusion-test-workers-* ENOTEMPTY, while the named files passed in isolation and then under the package lane with the broad-run worker budget. The rescue hardened the shared worker-root teardown's bounded ENOTEMPTY/EBUSY retry window and added explicit cleanup-invariant coverage, then removed the same-day core quarantine entries in ledger/config lockstep after proving the unexcluded package lane. Reusable pattern: when multiple core files fail with a shared worker-root cleanup signature, fix or prove the shared cleanup seam first; only quarantine residual files after the loaded unexcluded core lane still fails without a seam fix.
2026-06-18 engine isolation rescue (FN-6610): a full @fusion/engine lane reported unrelated expectation drift, vanished-cwd/git-config errors, and SQLite unable to open database file failures. The reusable isolation fix is to revalidate the shared test cwd/HOME/worker-root seam at the operation boundary: subprocess wrappers recreate the owned worker root, HOME, and cwd immediately before git, direct SQLite setup helpers recreate their redirected .fusion parent before DatabaseSync, and regression coverage removes the redirect sink/HOME/cwd mid-test before proving mkdtemp, SQLite open, and git config all still work. Do not mask this class with retries, worker reductions, or timeout bumps; quarantine only residual files after the shared seam and direct-open parents are proven under package load.
2026-06-19 CLI affected-lane rescue (FN-6734): a broad @runfusion/fusion lane reported default 5s test-body timeouts and fusion-test-workers-*/fixture ENOTEMPTY cleanup noise while isolated files exposed closeable real-store handles and a runtime-dist mock that was sensitive to package-lane module graph ordering. The rescue closed each real TaskStore/AgentStore before removing its temp fixture, kept task-list truncation coverage under the default timeout by reducing filler size rather than assertions, and preloaded the built @fusion/core barrel with vi.importActual before vi.doMock so complete dist artifacts exercise the CLI surface while partial stale dist skips cleanly. Prove this class with targeted file runs, pnpm --filter @runfusion/fusion test, the timeout-appeasement guard, bounded temp-prefix cleanup output, and the normal workspace gate/build; leave the CLI quarantine array empty when no file is actually quarantined.
2026-08-16 PostgreSQL first-sighting diagnosis (FN-9125): First classify each file by its real dependency path, not nearby failure timing. Retain complete tee output for repeated package lanes and record fan-out, server capacity, and target identities. If a current run cannot link a PostgreSQL test assertion to golden-template, DDL, pool, or teardown evidence, do not add timeouts/retries or infer a harness change: obtain CI/host activity and phase timings. A core PostgreSQL file is escalated rather than quarantined because quarantinedCoreTests remains policy-pinned empty; a non-PG engine file follows normal ledger-plus-exclude lockstep.
PostgreSQL teardown diagnostics
Set FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS=1 only while investigating an integration-test teardown. The default is off: no timer, probe connection, query, or sink file is created. When enabled, the shared PG harness records store.close, layer.close, adminSql.end, dropDatabase, and rmRootDir timings. FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_THRESHOLD_MS (default 2000) arms each phase watchdog; FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_HOOK_WATCHDOG_MS (default 12000) covers the whole teardown, safely below the inherited 15s hook budget. These are diagnostic observation bounds, never timeout extensions: a watchdog fires while work is still pending, so an aborted/hung teardown has evidence rather than only a post-hoc silence.
On a watchdog breach, the harness lazily opens one dedicated maintenance connection and queries pg_stat_activity across all databases. FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_PROBE_TIMEOUT_MS (default 1500) bounds connect/query/close; FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_MAX_PROBES (default 3) caps probes per process. Probes are single-flight, use no runtime or admin pool, are force-closed on abort, and are never awaited by teardown work. Timers are unref'd and every diagnostics failure is fenced, so the observer cannot keep a worker alive or change the outcome it measures.
Set FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_LOG=/path/to/teardown.jsonl to synchronously append one JSON object per record. The schema includes timestamp, pid/worker, trigger (phase-complete, phase-watchdog, teardown-watchdog, or snapshot), phase-duration map, thresholds, phaseIncomplete/elapsedAtSnapshotMs, probe outcome/suppression, and snapshot rows (datname, state, wait event, backend type, query age, and backend count). Sink errors only fall back to [pg-teardown-diagnostics] stderr output. For the next loaded sighting, preserve full output and structured evidence with:
FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS=1 \
FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_LOG=/tmp/fn-9127-diag-core.jsonl \
VITEST_MAX_WORKERS=12 \
pnpm --filter @fusion/core test 2>&1 | tee /tmp/fn-9127-core.log
Checkpoint parsed JSONL and run metadata in a durable task document after every run; /tmp is scratch storage, not the evidence system of record.
For a pre-registered loaded-flake campaign, record PostgreSQL version, max_connections, reserved slots, baseline and peak backend counts, and each run's selected file list. Track each registered test or hook independently: a capture satisfies only that exact identity; every uncaptured identity requires the full pre-registered subject-containing run set. Preserve all runner output and JSONL as task attachments as well as checkpointing parsed counters in task documents. Do not count a configured lane that does not select a subject as negative evidence for that subject.
PostgreSQL loaded-failure census
Use scripts/pg-loaded-failure-census.mjs to inspect an already-retained Vitest runner log and its optional teardown-diagnostics JSONL. The script never opens a cluster or runs tests. Supply --log, --diagnostics, and the recorded --ordinary-slot-ceiling; repeat --subject to label campaign files without dropping them. Its output contains total and failing files, lifecycle and failure-shape breakdowns, observed backend peak/headroom, wait histogram, phase-duration order statistics, and watchdog/probe-degradation counts.
A run is insufficient-data when its runner log lacks a complete Test Files summary or when the reported failed-file count cannot be reconciled to parsed failure blocks. A complete passing summary instead produces status: "measured" with failingFileCount: 0; never treat that zero as missing evidence or coerce incomplete input to a healthy result.
PostgreSQL timeout-boundary observer
FUSION_PG_TEST_TIMEOUT_BOUNDARY_OBSERVER=1 enables a diagnostic-only JSONL channel; when unset it creates no timer, probe connection, sink write, listener, or environment mutation. observeBoundary() wraps harness-owned setup and teardown work. openBoundary()/closeBoundary() span the shared harness's separate beforeEach and afterEach hooks, so shared.body measures test bodies without editing consumers. Each open body window receives a unique record joinKey; its stable file-level supersessionKey only closes a stale handle. Closing is idempotent; an abandoned window is superseded by a later open or disposed without inventing a completion.
The observer arms an unref'd watchdog at boundary start. Only a pending watchdog dispatches a maintenance-connection probe; completion records are host-only and emit when elapsed time meets FUSION_PG_TEST_TIMEOUT_BOUNDARY_OBSERVER_THRESHOLD_MS (default 2000; 0 measures every completion). With FUSION_PG_TEST_TIMEOUT_BOUNDARY_OBSERVER_LADDER_MS above zero, an independent unref'd checkpoint timer synchronously emits keyed payload-free kind:"progress" rows and rearms at that spacing. The last progress row is an elapsed lower bound for an abandoned boundary, with the configured spacing as its resolution; it is never a PostgreSQL-state snapshot. A settle/reject emits a keyed kind:"terminal" outcome without error text and clears both timers exactly once.
The watchdog first synchronously appends a keyed kind:"breach", payloadFree:true record, then appends its enriched kind:"watchdog" probe result. This two-phase form preserves a joinable elapsed boundary when Vitest abandons work during the watchdog-to-probe residual window; a payload-free breach never affirms a cluster-state mechanism. A dispatched probe survives boundary settlement and appends when it resolves, with settledDuringProbe, probeLatencyMs, and optionally probeStartDelayMs; records can therefore be out of order and consumers must join by the per-window joinKey, never by JSONL order or the shared supersessionKey. flush()/dispose() drain bounded pending and queued probes; a missed drain becomes probeSuppressed:"drain-timeout". The enabled-only beforeExit hook performs the same best-effort drain.
The independent, tighten-only bounds are statement timeout < probe timeout < inherited budget, queue timeout < probe timeout, and threshold <= per-boundary watchdog < inherited budget. A probe timeout is deliberately not tied to a short watchdog: forced wiring runs need a fast watchdog and a real cluster round trip. Floors are MIN_WATCHDOG_MS, MIN_PROBE_TIMEOUT_MS, and MIN_STATEMENT_TIMEOUT_MS; an impossible floor yields bounds-floor, never a wider budget.
| variable | default | purpose |
|---|---|---|
..._LOG |
unset | JSONL destination |
..._THRESHOLD_MS |
2000 | completion-record threshold; 0 measures all |
..._WATCHDOG_MS, ..._WATCHDOG_SETUP_MS, ..._WATCHDOG_BODY_MS, ..._WATCHDOG_TEARDOWN_MS |
12000 | global and per-boundary watchdogs |
..._PROBE_TIMEOUT_MS / ..._STATEMENT_TIMEOUT_MS |
1500 / derived below probe | bounded client/server probe stack |
..._PROBE_DRAIN_TIMEOUT_MS |
3000 | bounded flush()/dispose() drain |
..._MAX_CONCURRENT_PROBES |
1 | counting limiter, capped by MAX_CONCURRENT_PROBES_CEILING (8) |
..._PROBE_QUEUE_TIMEOUT_MS |
0 | bounded slot wait; a queued probe records delay |
..._MAX_PROBES |
4 | per-process diagnostic cost cap |
At concurrency 1 the cost profile matches the older single-flight observer. probeSuppressed:"concurrency" supersedes legacy single-flight; neither is a valid forced-wiring payload. The cap is configurable because once probes survive settle, a strict single flight would suppress simultaneous setup/body/teardown breaches. Probe payloads contain host load/CPU and watchdog scheduling drift (eventLoopLagMs, measured as the monotonic callback delay past its deadline at the snapshot instant), active SQL and lock/blocking rows, backend count, and golden-template owner/readiness plus separately reported granted advisory holders and non-granted advisory waiters. Only a non-owner waiter supports a template-convoy attribution; a holder alone does not.
For an enabled-wiring check, first run disabled and normal enabled bounds to compare duration, then threshold 0 to verify setup, shared.body, and teardown completion records, then force per-boundary watchdogs below measured elapsed times while independently retaining a probe/drain timeout above cluster round-trip and raising probe cap/concurrency to at least 3. Every forced boundary must have a non-suppressed payload; cap, concurrency, bounds-floor, drain-timeout, and error require diagnosis, not acceptance. Files using the shared harness are body-observable; direct createTaskStoreForTest files deliberately have no harness-owned body bracket and must be reported as body-unobservable.
Pass --boundary-observer <jsonl>, --vitest-json <reporter.json>, --body-unobservable-files <list>, and --fully-unobservable-files <list> to scripts/pg-loaded-failure-census.mjs. Capture the reporter per run with --reporter=dot --reporter=json --outputFile=<reporter.json>; it provides per-test duration and file identifiers but Vitest 3 provides no hook duration or failure position, so a hook-clock offset is unmeasurable rather than a point estimate. The census tolerates malformed and out-of-order rows, joins ladder/breach/enriched rows by key, and reports joined, attributed-by-ladder, body-unobservable, position-unobservable, and unjoined coverage counts plus probe suppression and latency distributions. afterEach consumer hooks are position-unobservable because the shared-harness body bracket closes before those hooks run. Failures from fully-unobservable files remain unjoined but are counted and listed separately so the known no-harness limitation is never mistaken for a missing observer join. Missing/empty input stays explicitly absent rather than a measured zero.
PostgreSQL campaign cluster-hygiene report
Before every FN-9152/FN-9153 campaign run, capture the maintenance-cluster rows and retain the three capture files plus the JSON report in the task document. The four counted hygiene classes are fusion_test_%, fusion_schema_template% per-module templates, fusion_schema_template_<pid>_golden<token> golden templates, and fusion_pool_%. The report classifies names itself, so the deliberately conservative SQL patterns cannot silently hide an unexpected matching name.
A capture is an envelope, not bare psql -qAt output: the # fusion-hygiene-capture v1 banner, kind, cluster identity, ISO timestamp, query identity, declared row count, body, and terminal # end make rows: 0 a measured zero. Missing, empty, header-less, truncated, count-mismatched, or wrong-kind captures are insufficient-data, never clean evidence. A supplied-but-unmeasured or wrong-kind --markers/--liveness companion makes the whole report insufficient-data; an omitted companion is not evidence, so Path A/C/D clean evidence requires all three captures supplied, measured, and bearing the same cluster identity. A stale marker row is dirty terminal evidence even when all four database counts are zero. With PG_URL="${FUSION_PG_TEST_URL_BASE:-postgresql://localhost:5432}", capture the database and marker rows as follows (write the displayed envelope fields around each query result):
capture() { # capture KIND QUERY SQL OUTPUT
kind="$1" query="$2" sql="$3" output="$4"
cluster="$(psql "$PG_URL/postgres" -qAt -c "SELECT current_setting('port') || '|' || version() || '|' || current_database();")"
rows="$(psql "$PG_URL/postgres" -qAt -c "$sql")"
{ printf '%s\n' '# fusion-hygiene-capture v1' "# kind: $kind" "# cluster: $cluster" "# captured_at: $(date -u +%Y-%m-%dT%H:%M:%SZ)" "# query: $query" "# rows: $(printf '%s\n' "$rows" | sed '/^$/d' | wc -l | tr -d ' ')"; printf '%s\n' "$rows"; printf '%s\n' '# end'; } > "$output"
}
capture databases leftover-v1 "SELECT datname, pg_get_userbyid(datdba), CASE WHEN datistemplate THEN 't' ELSE 'f' END, (SELECT count(*) FROM pg_stat_activity a WHERE a.datname = d.datname) FROM pg_database d WHERE datname LIKE 'fusion_test_%' OR datname LIKE 'fusion_schema_template%' OR datname LIKE 'fusion_pool_%' ORDER BY 1;" databases.capture
capture markers markers-v1 "SELECT name, created_at FROM public._fusion_golden_templates ORDER BY 1;" markers.capture
# For each parsed template owner PID, record exactly one `<pid>|alive` or `<pid>|dead` body row from `ps -p "$pid" -o pid=` in an analogous `kind: liveness`, `query: liveness-v1` envelope.
node scripts/pg-cluster-hygiene-report.mjs --databases databases.capture --markers markers.capture --liveness liveness.capture --json
The report is advisory: reclaimable-dead-owner means only recorded dead liveness plus zero sessions; missing liveness, an alive owner, an unparseable name, or sessions retains the row. It never grants approval. Removal requires all of G2–G6: that recorded liveness verdict, immediate zero-session recheck, one literal name-scoped DROP DATABASE without FORCE against a present target, equality-only deletion of that same marker row (never a broad stale-marker sweep), and a human-authored, verbatim, action-explicit, evidence-informed approval recorded after the pre-state evidence in the task's removal-approval document. Agents may never self-issue or infer that approval. A refused drop, pre-check abort, partial marker reconciliation, or unmeasurable/dirty post-state is terminal: record the evidence, file or reuse one scoped follow-up, exit honest-blocked, and never retry, force, or claim clean. Record the four counts for every campaign run; an absent report is FN-9152 R3 leftover-counts-unrecorded evidence, not a clean arm.
PostgreSQL DDL loaded-lane acceptance metric
Use scripts/pg-ddl-lane-metric.mjs before judging a PostgreSQL DDL structural candidate. Run at least seven interleaved control/candidate pairs at VITEST_MAX_WORKERS=12; preserve one diagnostics JSONL sink and complete runner log per invocation. The exact lane is:
FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS=1 \
FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_LOG=/tmp/fn-9134-control-01.jsonl \
VITEST_MAX_WORKERS=12 \
pnpm --filter @fusion/core exec vitest run src/__tests__/postgres --reporter=dot 2>&1 \
| tee /tmp/fn-9134-control-01.log
Alternate the default control and enabled candidate; do not repeat an unfavorable completed run. The primary metric is median complete lane wall-clock time. Every run must be green and report zero surviving fusion_test_% databases. Completed dropDatabase p95/max, watchdog counts, and teardown-watchdog counts are descriptive only: moving cleanup out of the hook can zero them by construction.
The pre-registered verdict is improved only when each lane has at least seven valid samples, the candidate median is below the control's 25th percentile, and the candidate worst wall time does not exceed the control median. Any red run, missing wall time, or non-zero leak count is no-improvement; fewer than seven samples in either lane is insufficient-data. Generate a report without contacting the cluster:
node scripts/pg-ddl-lane-metric.mjs \
--run control:/tmp/fn-9134-control-01.jsonl:/tmp/fn-9134-control-01.log:0 \
--run candidate:/tmp/fn-9134-candidate-01.jsonl:/tmp/fn-9134-candidate-01.log:0
2026-06-19 quarantine audit (FN-6740): the 2026-06-19 ledger batch expires on 2026-07-03. FN-6740 found no ledger/config half-state and chose no inline rescue/delete. The five CLI files (extension-goal-tools, extension-mission-goal-tools, extension-task-tools, extension, research-extension-tools) are explicitly deferred to FN-6734's outcome and must not get a duplicate rescue task. Five core files (activity-analytics, db, store-create-summarize-deferred-hook, vitest-teardown-worker-root-cleanup, settings-export) were rescued by FN-6741 after the loaded @fusion/core lane passed with only store-concurrent-writes re-quarantined; settings-export now closes its TaskStore before fixture cleanup. The dashboard files were split by likely root cause: FN-6742 rescued session-cross-tab cleanup ENOTEMPTY by closing the route/task-store seam before fixture removal; FN-6743 owns the third-repeat QuickEntryBox focus-restoration race after FN-6514/FN-6642; and FN-6744 rescued WorkflowNodeEditor duplicate-merge-seam concurrency by making fragment seam conflicts consult the loaded workflow IR before React Flow canvas nodes finish materializing. Until the remaining dashboard and core follow-ups rescue with root-cause evidence or delete under the ratchet, leave all corresponding ledger entries and package excludes in lockstep.
2026-06-19 core suite-load rescue (FN-6741): activity-analytics.test.ts, db.test.ts, store-create-summarize-deferred-hook.test.ts, vitest-teardown-worker-root-cleanup.test.ts, and settings-export.test.ts were rescued before their 2026-07-03 deletion deadline. The key evidence was a loaded pnpm --filter @fusion/core test pass across the package after re-quarantining store-concurrent-writes.test.ts, with no ENOTEMPTY, EBUSY, hook timeout, or missed deferred hook in the rescued files. Four files needed no weakening because their regression value still held under load; settings-export.test.ts kept its import/export coverage but now closes the real TaskStore before removing the fixture root. store-concurrent-writes.test.ts remains in the deletion ratchet after merge verification re-observed the broad-lane SQLite lock flake. Required closure evidence for this class is ledger/config convergence, the rescued package lane, the timeout-appeasement guard, pnpm test:gate, pnpm test, pnpm typecheck, and pnpm build.
2026-06-20 core task-documents rescue (FN-6790): packages/core/src/__tests__/task-documents.test.ts stays loaded and unquarantined. The broad-lane symptom was an ENOENT during atomic task.json rename; the root-cause class is fire-and-forget TaskStore deferred task-created work (title summarization and task-created hook) entering an update after store.close() while the fixture root is being removed. The fix tracks active post-summarization write/hook work, makes close() mark the store as closing and await active work, and skips late deferred work that has not entered the write phase so intentionally stalled summarizers do not hang teardown. The regression test releases a controlled deferred summarizer only after close and root removal, then asserts the fixture root is not recreated. Closure evidence is targeted file coverage, a loaded pnpm --filter @fusion/core test pass, timeout-appeasement guard, bounded fusion-test-workers-*/kb-task-docs-test-* output, pnpm test:gate, pnpm test, pnpm typecheck, and pnpm build; no quarantine ledger/config entries or timeout/worker appeasement are allowed for this file.
2026-06-20 residual CLI quarantine triage (FN-6795): the six 2026-06-19 residual entries were temporarily unexcluded and exercised under targeted and loaded lanes. store-concurrent-writes.test.ts, extension-goal-tools.test.ts, extension-mission-goal-tools.test.ts, and research-extension-tools.test.ts were rescued because their direct and package/gate lanes stayed green with no ENOTEMPTY, lock exhaustion, or cross-test state drift. The final full @runfusion/fusion lane still timed out extension-task-tools.test.ts, extension.test.ts, and a newly observed bin.test.ts case only under package load while the focused rerun passed, so those files remain quarantined in ledger/config lockstep with the original 2026-07-03 deletion deadline for the two 2026-06-19 residuals. Treat future work as a fixture-load root-cause search, not timeout/retry/worker appeasement.
2026-06-21 retained CLI quarantine rescue (FN-6839): bin.test.ts, extension-task-tools.test.ts, and extension.test.ts were rescued before their 2026-07-03/2026-07-04 deletion deadlines. The failed prior attempt to skip task:created hooks ruled out a hook-only root cause; the real reusable invariant is that TaskStore.close() is async and must be awaited for both extension cached stores and direct fixture stores before removing temp roots, otherwise deferred filesystem work and SQLite/WAL handles can survive under loaded @runfusion/fusion workers. closeCachedStores() now awaits each cached store close, the quarantined fixtures await direct/cached shutdown, and the extension regression test asserts cached shutdown does not resolve before async close settles. The three ledger entries and packages/cli/vitest.config.ts excludes were removed in lockstep; the grouped three-file lane and full CLI package lane pass unexcluded with no hook/body timeout, no ENOTEMPTY/EBUSY, and no timeout/retry/worker appeasement. The broader pnpm test command is currently blocked before tests by unrelated line-count guardrail failures tracked by FN-6849, not by these rescued CLI files.
2026-06-19 dashboard session-cross-tab rescue (FN-6742): packages/dashboard/src/__tests__/session-cross-tab.test.ts was rescued before its 2026-07-03 deletion deadline. The loaded dashboard-api-quality-backfill shard reproduced the original fusion-test-workers-* ENOTEMPTY cleanup failure with the quarantine exclude temporarily removed, while the test's assertions retained value by failing when the expected lock holder was mutated from tab-a to tab-z. The fix keeps the test unquarantined by disposing the created API router, stopping AiSessionStore scheduled cleanup, closing the real TaskStore/SQLite handles, hiding route EventEmitter hooks not used by this harness, and draining four bounded check-phase turns before deleting the temp root. The ledger and packages/dashboard/vitest.config.ts exclude were updated in lockstep; later loaded runs no longer failed this file, and unrelated dashboard loaded-suite failures are tracked separately rather than weakening this test.
2026-06-21 dashboard quarantine lockstep cleanup (FN-6860): packages/dashboard/src/__tests__/dev-server-process.test.ts and packages/dashboard/src/__tests__/session-cross-tab.test.ts were intended to be cleared from the deletion ratchet after repeated dashboard-api-quality-backfill loaded-shard runs passed with the excludes removed. dev-server-process kept its process-lifecycle regression value by tracking lifecycle generations, disposed state, active stdout/stderr line work, and fallback probe work before close/failure cleanup resolves; its tests now assert duplicate URL detection is suppressed and probe timers are cleared on failure/restart/cleanup. session-cross-tab needed no code change in this batch because it was already active in Vitest config, but FN-6937 later found the stale ledger-only entry still present at HEAD. Closure evidence for this class is the grouped rescued-file lane, full test:quality:api:backfill runs, ledger/config empty-state convergence, lint, gate, pnpm test, and build, with no timeout/retry/worker appeasement.
2026-06-22 stale ledger-only dashboard cleanup (FN-6937): packages/dashboard/src/__tests__/session-cross-tab.test.ts was already active because packages/dashboard/vitest.config.ts had no quarantine exclude. FN-6937 reconfirmed the rescue under the loaded dashboard-api-quality-backfill shard, mutation-tested the lock-holder assertion by changing tab-a to tab-z and observing the expected failure, reverted the mutation, reran the loaded shard cleanly, and then removed the stale ledger-only row from scripts/lib/test-quarantine.json. Required closure evidence is ledger/config convergence, no session-cross-tab ledger match, the loaded backfill shard, the timeout-appeasement guard, bounded temp-prefix output showing no kb-session-cross-tab-* roots, pnpm lint, pnpm test, and pnpm build.
2026-06-19 dashboard WorkflowNodeEditor rescue (FN-6744): packages/dashboard/app/components/__tests__/WorkflowNodeEditor.test.tsx was rescued before its 2026-07-03 deletion deadline. The original duplicate-merge test passed in isolation but was load-sensitive because handleInsertFragment derived existing seams only from transient React Flow nodes; a fast palette click could arrive after activeWorkflow.ir loaded but before the canvas nodes materialized, allowing an invalid duplicate merge seam instead of showing the conflict alert. The fix keeps the test unquarantined by treating IR merge nodes as the merge seam and by unioning seams from the loaded IR only during initial canvas materialization, preserving post-load canvas-state semantics. Regression coverage now exercises both desktop and mobile fragment insertion surfaces and asserts the conflict affordance appears without growing the rendered graph. The ledger and packages/dashboard/vitest.config.ts exclude were removed in lockstep; targeted file runs, repeated test:quality:app:components-b, lint, gate, typecheck, and build are the closure evidence. A broader @fusion/dashboard test run currently fails unrelated Command Center ProductivityArea mock drift tracked by FN-6754, so do not re-quarantine WorkflowNodeEditor for that lane.
2026-06-19 dashboard API shard isolation (FN-6753): packages/dashboard/src/__tests__/routes-auth.test.ts is classified as suite-load coupling. The observed symptom was a timeout only under the broad dashboard-api-quality-backfill shard; five loaded local runs of the isolated shard did not expose a concrete teardown, probe-spy, or product-code root cause. The remedy is shard isolation, not quarantine: keep routes-auth in the curated dashboard-api-quality include list so authentication coverage stays active, and let backfillApiExclude remove it from the broad src/**/*.test.ts backfill glob. Use the same pattern for critical route suites that fail only under broad backfill contention after loaded local proof cannot identify an owned fixture seam: preserve coverage in a curated shard, document the classification, and avoid timeout bumps, retries, worker reductions, or ledger entries unless a later loaded run proves a real flaky file that needs the deletion ratchet.
2026-06-16 rescue (FN-6514): packages/dashboard/app/components/__tests__/QuickEntryBox.test.tsx was rescued before its 2026-06-30 deletion deadline. The file still caught real quick-entry behavior regressions, but it leaked jsdom descriptors for window.innerWidth, window.matchMedia, document.visibilityState, URL.createObjectURL, and URL.revokeObjectURL; a mobile viewport helper could leave later tests in the same dashboard backfill shard observing innerWidth=375 and mismatched responsive assertions. The rescue removed the ledger/config quarantine entries in lockstep, captured each original PropertyDescriptor at module load, restored those descriptors (or deleted own properties that were originally absent) in afterEach, and added a guard test that mutates all rescued globals before asserting they return to their original descriptors. Reusable pattern: any test file that changes jsdom globals with Object.defineProperty or spies on replaceable globals must snapshot the original descriptor at the top of the file, restore it in every afterEach, and prove the invariant with a guard test; do not use timeout bumps, retries, worker changes, or blanket vi.restoreAllMocks() when module mocks depend on stable implementations.
Gate eviction: a flake inside the merge gate cannot block all merges while red — it is evicted by removing its line from the engine-core allow-list (no quarantine entry needed unless it should also leave the non-blocking tier).
Gate admission: the mirror operation — add the test's path to the engine-core include array in packages/engine/vitest.config.ts, citing the evidence of value (a real regression it caught) in the PR. Keep the project under its ~60s wall-clock budget.
Product-race escalation: a second quarantine in the same subsystem is a smell that the flake is a real product race, not test noise — look at the product code before deleting (a dashboard flake was "stabilized" three times before being found to be a real race; see docs/solutions/ui-bugs/skill-autocomplete-highlight-reset-on-swr-revalidation.md).
CI shard balancing (duration-weighted)
scripts/ci-test-shard.mjs packs the 4 CI shards (pnpm test:ci:shard --shard N --total 4,
called from full-suite.yml, non-blocking) by measured duration, not test-file count, using the
committed scripts/test-timings.json snapshot (U1/R4). A package's weight is the sum of
its files' recorded durations; files (or whole packages) absent from the snapshot fall
back to the snapshot's median per-file duration so untimed packages weigh
commensurably. Untimed packages are named in a logged warning.
- Engine keeps
vitest --shard X/Yvirtual slicing (itstestis a single vitest invocation:--project=engine-default --project=engine-reliability); slices are now weighted by duration. - Dashboard is not
--shard-sliced — its defaulttestscript is a bounded concurrent lane orchestrator, so a forwarded--shardcannot apply coherently. Instead each leaf quality lane (enumerated programmatically frompackages/dashboard/package.jsonand the dashboard quality orchestrator) is a separately-weighted schedulable unit; a shard runspnpm --filter @fusion/dashboard run <lane>for its assigned lanes. Every lane is assigned to exactly one shard. Lane weight is the sum of durations of the files the lane's--projects execute, derived from the vitest config projectinclude/excludeglobs (imported viatsx); if the config cannot be imported the package duration is apportioned evenly across lanes (logged aseven-apportionment). - Inspect the plan without running it:
node scripts/ci-test-shard.mjs --dry-run --total 4(optionally--shard N) prints the plannedpnpmcommands and per-shard weight totals. - Measure per-process startup cost:
node scripts/ci-test-shard.mjs --cold-start-probe <package-name>runs the package's cheapest test file in isolation and reportswall − test timeoverhead (the signal behind the deferred vitest-4 upgrade gate).
Snapshot staleness policy
The snapshot carries capturedAt. If it is older than 30 days, the planner prints a
prominent warning and proceeds (balance degrades gracefully toward the file-count status
quo, never below it) — it does not fail the build. Refresh is manual/scheduled from
the default branch only: each CI shard uploads per-shard JSON timing artifacts (U1), and
node scripts/ci-test-shard.mjs --write-timings merges them into the snapshot. Download the
shard artifacts into .timings/ first (the default lookup directory), or pass
--inputs-dir <path> to point at wherever they were downloaded.
When deleted test files leave phantom snapshot entries, run
node scripts/ci-test-shard.mjs --prune-timings. Pruning removes only paths absent from disk,
drops packages with no remaining paths, and never restamps capturedAt. A real refresh still
requires --write-timings with timing artifacts, and the 30-day staleness budget still applies. A future
scheduled job can gate on freshness via node scripts/ci-test-shard.mjs --check-timings-staleness,
which exits non-zero when the snapshot is missing or older than the 30-day budget. Package test
wrappers that launch Vitest (including @fusion/desktop) must forward caller reporter/output
arguments unchanged: shard telemetry uses --reporter=json plus a package-relative
--outputFile.json=.timings/timings-*.json. Wrappers may default to dot only when callers do
not select a reporter; a successful artifact set must include parseable, non-empty desktop
testResults under packages/desktop/.timings/.
Weekly test velocity baseline
FN-6612 tracks feedback-loop velocity as signal-per-second, not as a new blocking gate. Refresh the weekly baseline from a clean worktree with:
pnpm test:velocity -- --measure --write-report
In --measure mode, the script first runs a non-measured build preflight (pnpm build) so the built CLI and workspace dist artifacts exist before any lane is timed. The preflight duration is setup cost and is excluded from pnpm test:gate, pnpm smoke:boot, and pnpm test history fields; if the preflight fails, the report records Build preflight (pnpm build) in Measurement failures instead of fabricating lane times or letting boot smoke appear unavailable. Use --skip-build-preflight only in CI or another environment that has already built the workspace.
After the preflight, the script runs pnpm test:gate, pnpm smoke:boot, and pnpm test with bounded async process supervision, then appends the measured row to scripts/test-velocity-history.json and rewrites the postable artifact at docs/test-velocity-baseline.md. It reads the slowest 20 files from the committed scripts/test-timings.json snapshot and the flake/quarantine count plus 14-day deletion-clock buckets directly from scripts/lib/test-quarantine.json; do not run the full suite just to populate the slowest-file table.
Use cheap report-only regeneration when measurements already exist:
pnpm test:velocity
Each week, copy the Post to #leads block from docs/test-velocity-baseline.md. If a measured command fails because the local environment is not ready, keep the failure recorded in the report instead of fabricating a time, then fix or rerun separately as appropriate. Do not wire pnpm test:velocity, test:full, or any slow-suite expansion into PR checks; the merge gate stays the thin Lint, Typecheck, Build, and Gate path.
The baseline document is generated; never hand-edit it. Attach a durable measurement verdict with pnpm test:velocity -- --note "<text>" [--note-target <capturedAt|ISO-cycle>]. Notes are persisted on their history entries and the report renders all annotated cycles newest first with no history window cap, so an old investigation remains visible after later weekly appends. --note-target must match exactly one entry; use its capturedAt timestamp when an ISO week would be ambiguous.
Targeted commands
pnpm --filter @fusion/core test
pnpm --filter @fusion/engine test
pnpm --filter @runfusion/fusion test
pnpm smoke:pipeline
pnpm test:scripts
node --test scripts/__tests__/*.test.mjs
For a single Vitest file, use package-local exec vitest:
pnpm --filter @fusion/core exec vitest run src/__tests__/central-db.test.ts --silent=passed-only --reporter=dot
Changed-only test cache (pnpm test)
pnpm test runs scripts/test-changed.mjs, which selects only the workspace
packages affected by your branch diff (plus their reverse-dependents) and skips
packages whose content hasn't changed since they last passed. A per-package
pass-cache lives at node_modules/.cache/fusion/test-cache.json.
To see which mode a run would pick — and why — without running any tests:
node scripts/test-changed.mjs --print-mode prints the
[test-changed] mode=… reason=… packages=… decision line and exits.
What a cache entry's hash covers (dependency-aware invalidation)
Each package's cache hash (computePackageHash) folds in, so any of these
changing forces that package to re-run:
- The package's own tracked files, hashed via the working-tree bytes for any file that is dirty (unstaged/uncommitted edits) or untracked-not-ignored, and via git's index blob SHA only when the file is fully clean. This means an unstaged edit to a tracked file busts the cache — no false HIT on a stale index blob.
- Every transitive workspace dependency's own hash. A change to
@fusion/coreinvalidates the cache entries ofengine,dashboard,cli, and everything else that (transitively) depends on it, even when the dependent's own files are untouched. This is the R11 correctness fix: a dependent is never cache-skipped when a dependency it consumes has changed. - Shared inputs folded into every package:
pnpm-lock.yaml,tsconfig.base.json, and the sharedpackages/core/src/__test-utils__tree. The test-utils tree is imported by nearly every package's vitest config via a relative cross-package path, including packages that have no@fusion/coreworkspace dependency (mobile, droid-cli, pi-*, and the plugins). Folding it in globally (liketsconfig.base.json) guarantees an edit there invalidates the whole workspace.
The hash carries a version prefix (HASH_VERSION_PREFIX). Bumping it (done in U4:
v1 → v2) invalidates every pre-existing entry exactly once; old-format cache
files are discarded gracefully rather than crashed on.
Escape hatches
If you suspect a stale or wrong cache result (e.g. a flaky test that happened to pass got cached, or you want to force a clean re-run), bypass the cache:
pnpm test --no-cache # bypass cache reads AND writes for this run
FUSION_TEST_NO_CACHE=1 pnpm test
--no-cache re-runs every selected package without consulting or clearing the
cache file; a subsequent normal pnpm test still hits the cache. pnpm test:full
already passes --no-cache (a full run means full). These flags already exist;
this section documents them.
TTL rationale (7-day expiry)
Entries older than 7 days are treated as a MISS even on a hash match
(CACHE_MAX_AGE_MS). The TTL is intentionally retained even though dep-aware
hashing makes content-staleness impossible: it guards against environmental
drift that the content hash cannot see — toolchain/Node upgrades, OS or native
dependency changes, and other host-level shifts that can change test outcomes
without changing any hashed file. Seven days bounds that blind spot while keeping
the cache useful across a normal work week.
Engine test helper convention
packages/engine/src/__tests__/executor-test-helpers.ts defaults both isUsableTaskWorktree to true and classifyTaskWorktree to { ok: true } via a helper-level worktree-pool mock. To test failure paths, override with vi.spyOn(worktreePool, "classifyTaskWorktree").mockResolvedValueOnce({ ok: false, classification: "unregistered", reason: "..." }) (or isUsableTaskWorktree for legacy call sites). Production liveness assertions in executor.ts are unchanged.
Graph-owned executor fixtures: Construct TaskExecutor with createWorkflowRoutingAgentStore(store).agentStore; graph routing otherwise fails closed before any implementation or review seam is reached. The shared helper defaults reused-worktree preflight to reconcileSecretsEnvFingerprint → { executionSafe: true, outcome: "clean" } and refreshReusedWorktreeBase → { kind: "up-to-date", executionSafe: true, durableBaseSha: null }. Override either mock with its documented blocked result when testing WorktreeBaseRefreshError; do not invent result-union members. StepSessionExecutor owns forced/new-session execution, but resume-vs-fresh SessionManager assertions must use an unpinned, single-session workflow because step sessions never resume task.sessionFile.
Before reporting done
- Code changes: affected package tests + any directly relevant browser/build lane.
- Cross-package, shared test infrastructure, or CI changes:
pnpm test:full. - Production/bundling-sensitive changes:
pnpm build. - Staged CLI plugin core imports: run
pnpm --filter @runfusion/fusion exec vitest run src/__tests__/staged-plugin-core-imports.test.ts --silent=passed-only --reporter=dot. The AST guard checks every staged plugin's non-test source against the CLI core-runtime shim and each plugin manifest; it intentionally rejects unclassifiable, namespace-escaping, default, and core-subpath imports. - Substantial work:
pnpm verify:workspace. - If you skip a relevant lane, say why.
Test file organization
Test for src/foo.ts → src/__tests__/foo.test.ts. Test for app/components/Bar.tsx → app/components/__tests__/Bar.test.tsx. __tests__/ is the standard.
What NOT to write
Tests should cover behavior a user could notice break, not implementation shape. Don't write:
- CSS-class permutation tests — use one
it.eachfor the boolean matrix, not oneitper combination. - Field-presence tests when a payload-roundtrip test already exercises the same field.
- React.memo tautologies — testing
React.memotests React, not us. Test custom comparators directly, one case. - Mock-the-world wiring tests — if a test mocks 8+ deps just to render a component, shim children with
() => nullor delete and rely on an integration test one level up. - Structural CSS assertions — "tab uses .class-name not inline style". Consolidate into one aggregate layout-contract test per component.
Prefer it.each over copy-pasted it() blocks. When trimming, keep: first case + opposite case + any precedence/override case.
What TO keep unconditionally
- Tests linked to an FN-ticket in describe/it names — these guard real regressions.
- Integration tests exercising real SQLite, real worker pool, or spawned processes.
- Lean core/engine unit tests with low mock burden.
Test isolation for module-singleton state
When a test owns a process-wide singleton that has asynchronous owners (timers, processors, or lane starts), use the same teardown in beforeEach and afterEach: await every tracked owner’s stop() with Promise.allSettled, then clear all shared state in a finally block. Do not clear first: a pending stop or callback can repopulate the singleton after the apparent reset. Test reset mutators establish cleanup; read-only inspection seams must prove reservations, mutex/draining state, registrations, and companion module-global slots are actually empty. Fix the isolation seam at the root rather than adding retries, wider timeouts, weakened assertions, or a quarantine entry.
Cross-package Vitest mock scoping
In this pnpm workspace, one dependency version can resolve to several peer-hashed instances. A vi.mock declared in a packages/cli test is keyed by the resolved module path, so it does not automatically reach @fusion/dashboard or @fusion/engine source that imports another instance. For @earendil-works/pi-coding-agent, retain the anchored package-root alias in packages/cli/vitest.config.ts (with subpath aliases ordered first). The symptom is a mocked function with zero calls alongside real runtime/provider log noise. Use an importOriginal()-spread factory when cross-package consumers need unmocked exports, and keep a guard that fails when the alias is removed.
Standing Rule: Do Not Add Slow Tests (FN-5048)
- Default new tests to narrow seams, in-memory fakes, shared harnesses, and targeted assertions.
- For bug-fix regressions, also follow
AGENTS.md→ Standing Rule: Fix the Invariant, Not the Repro (FN-5893) so coverage proves the invariant across known surfaces, not just one repro. - Prefer fake timers over real polling/time waits (FN-2707 pattern: advance timers inside
act(...), restore withafterEach(() => vi.useRealTimers())). - Do not mask slowness by raising worker/concurrency knobs (
FUSION_TEST_TOTAL_WORKERS,FUSION_TEST_CONCURRENCY,VITEST_MAX_WORKERS, workspace concurrency settings). - Do not add net-new real-network calls, real-
setTimeoutpolling loops, or mock-the-world component shells when a narrower seam exists. - Real Pi SDK catalog tests in the engine package must use
src/__tests__/_model-runtime-fixture.ts: warm its shared runtime inbeforeAlland request a fresh registry rather than constructingModelRuntimeinside timed test bodies. - Use the canonical taxonomy in What NOT to write and What TO keep unconditionally when deciding trim vs keep.
- See
docs/test-speed-audit-FN-5048.mdfor the measured baseline offender list and optimization priorities.
Surface Enumeration checklist
Copy this checklist into a bug-fix or UI-affordance add/remove task's ## Surface Enumeration section and make the implementation tests prove the invariant across every checked surface. This checklist applies to bug-fix tasks and UI-affordance add/remove tasks that add, remove, or restructure icons, buttons, chevrons/arrows, toggles, badges, menu entries, or click targets. See AGENTS.md → Standing Rule: Fix the Invariant, Not the Repro (FN-5893) for the enforced planning/review contract.
- Providers / bridges / execution paths touched by the invariant
- For merge fixes: every merge-admission door, including periodic feeds, merge workers, direct/manual doors, and workspace landing
- For content-bound merge gates: singular diff fingerprints, workspace per-repository evidence, unavailable/empty descriptors, disabled review groups, renamed review lanes, and workflow-selection provenance (reader absent, no selection, and read failure)
- For merge finalization: confirmed-merge reconciliation, non-checklist blockers, and no failed park after a landed merge
- For execution/merge exclusion: live executor refusal with and without approval, reciprocal executor dispatch refusal during merge, in-flight review revocation, and final ref-advance recheck
- For worktree cleanup: active-session, successor-session after abort, raw/canonical path spellings, and workspace sub-repository worktrees
- Long-running subprocess or verification-active surfaces when the invariant involves engine liveness, stuck detection, or command execution (
fn_run_verification, configured commands, timeout/deadline behavior) - Desktop + mobile breakpoints / platforms that exercise the behavior
- Empty / undefined / duplicate / populated data states
- Shared hooks / components / modules / helpers reusing the logic
- Every component that renders the affordance (search the codebase for the icon/class/testid, not just the one the user pointed at)
- Leftover shells after removal — empty buttons, orphaned click targets, now-unused wrappers, dangling aria-labels — are explicitly checked and fixed/hidden
Motivating incident: FN-6115/FN-6118/FN-6123 — a single workflow-row chevron required three tasks to fully remove because the affordance rendered across multiple components and one mobile surface kept an empty btn-icon button shell.
Symptom Verification for bug-class tasks
Bug-class/bug-fix tasks must also include a ## Symptom Verification section so FN-5893 acceptance proves the original user-visible failure is gone, not merely that a change landed or broad checks are green. Feature/docs/non-bug tasks are not required to carry this section.
Use the exact heading ## Symptom Verification and include all three required contents:
- Original symptom — what the user/issue reported was broken.
- Exact reproduction — the precise steps, inputs, fixture, or automated repro that triggered the failure.
- Assertion it is gone — final verification reproduces the original failure condition and asserts it no longer occurs via a real automated test.
Symptom-based acceptance is mandatory for bug fixes: reproduce the original failure, prove it is gone, and keep the invariant covered across the ## Surface Enumeration checklist. Green build/tests alone are insufficient when they do not exercise the reported symptom.
Chromium touch hit-testing
Touch-resize regressions use the dashboard Vite fixture and Chromium CDP Input.dispatchTouchEvent start/move/end events. Run the dedicated dashboard-browser-touch Vitest project with:
pnpm --filter @fusion/dashboard test:touch-geometry
The shared task-modal/FloatingWindow lane in packages/dashboard/src/__tests__/task-modal-touch-resize-browser.test.ts covers 768px and wider tablets, a 767px phone sheet, and true phones on an ephemeral Vite port (port: 0); it must never bind port 4040. This is required where elementFromPoint and real touch hit testing matter; jsdom pointer dispatch does not provide layout hit testing.
This repository has no Playwright test runner: Vitest runs playwright-core as the Chromium CDP driver. Do not use --project chromium or browserName guidance from FN-8605/FN-8607. The lane finds an already-installed browser via FUSION_BROWSER_SMOKE_BROWSER, CHROME_BIN, or platform candidates, and skips with an explicit reason when none is available; it never downloads a browser. FUSION_DASHBOARD_DEEP=1 does not collect this spec a second time — the dedicated project remains its only collection lane.
PostgreSQL DDL admission bound
FN-9136 evaluated and rejected per-fork TRUNCATE database reuse: its two-sided golden-template gate passed the cluster smoke, but the required seven-pair 12-worker campaign left dead-fork pooled databases because the experiment lacked an awaited fork-exit flush and degraded to a shared local identity. The experimental wiring was removed rather than weaken isolation; see docs/solutions/test-failures/postgres-ddl-admission-bound.md.
FN-9130 evaluated a reusable advisory admission primitive: server session locks coordinate forks while a process-local ledger coordinates counted same-session locks within a fork; async context allows true nesting only. It uses one maintenance-database connection per fork, holds a slot for one statement only, and reports fail-open degradation. FUSION_PG_TEST_DDL_MAX_CONCURRENCY and FUSION_PG_TEST_DDL_ADMISSION_ACQUIRE_TIMEOUT_MS configure that primitive for its deterministic coverage.
The harness does not currently wire the primitive into CREATE DATABASE or DROP DATABASE. Uniform CREATE/DROP pooling regressed the 12-worker lane (49 watchdogs / 5,068ms versus a 4–5 / 3,284ms baseline); drop-only wiring also regressed (27 / 3,361ms). A bounded off-hook reaper (R=2/Q=8, flush, and dead-pid sweep) was then measured and reverted: its watchdog zero was structural, but green runs took 117.2s and 122.4s against a 108.1s baseline maximum and a later run timed out. FN-9136 subsequently evaluated and rejected candidate C (per-fork reuse plus TRUNCATE) because its seven-pair campaign leaked dead-fork pooled databases. The pg-gate's four-worker cap remains a separate lane-shape policy.
Workspace lifecycle parity regression
Use focused workspace tests for a two-repository task with one explicitly scoped, modified repository:
pnpm --filter @fusion/core exec vitest run src/__tests__/postgres/workspace-worktrees-concurrent-merge.pg.test.ts --silent=passed-only --reporter=dot
pnpm --filter @fusion/engine exec vitest run src/__tests__/reviewer-workspace.test.ts src/__tests__/self-healing-workspace.test.ts src/__tests__/workspace-merger.test.ts --silent=passed-only --reporter=dot
pnpm --filter @fusion/dashboard exec vitest run app/components/__tests__/WorkspaceWorktreesSummary.test.tsx --silent=passed-only --reporter=dot
The parity invariant is that a mono-repository task and a workspace task changing one scoped repository have identical review, completion, and landing outcomes. The acquired clean peer must be displayed as No changes — not reviewed, must not get a blocking verdict, and must not become a partial-land target.
Branch-writer validation regressions
When task-branch validation changes, test production acquisition and task creation callers in addition to the validation boundary. Keep a source census of direct and computed branch patches, and exercise single-repository assignment persistence failure plus workspace suppression so a caller cannot silently omit provenance.
Workspace review-to-land regression
Workspace merge regressions must use the production per-repository review capture before landing. Cover a modified scoped repository, a clean acquired peer (NOT_REVIEWED), approval-missing, content-changed, and out-of-scope changes; do not manufacture the landing fingerprint as the decisive happy-path oracle.