Quarantine test files consistently failing on the non-blocking full-suite
CI on main, per the AGENTS.md deletion-ratchet policy:
Engine-default (shard 1-2): ce-workflow-step-conventions,
executor-column-agent-principal, restart.integration,
scheduler-node-unreachable-audit, scheduler-overlap-starvation,
scheduler-ephemeral-toggle, user-configured-command-no-execsync
Engine-reliability (shard 1): lease-recovery-central-claim,
owning-node-unavailable-interactions, todo-inprogress-flapping
CLI (shard 3): extension.test.ts
Each has a matching entry in scripts/lib/test-quarantine.json with the
failing CI run link and quarantinedAt date. Tests will be deleted
after 14 days unless rescued with a root-cause fix.
Three changes to make `pnpm test` reliably minimal and fail gracefully:
- @fusion/core is now a memory-envelope/wide-fan-out package (was unguarded).
It's the hub nearly everything imports (~354 test files), so a core source
edit made `vitest --changed` expand to ~the whole core suite and blow past the
engine's 15-min verification kill -> SIGKILL + task restart. Adding it to
SCOPED_AFFECTED_MEMORY_ENVELOPES applies the wide-fan-out guard (run only
directly-changed core tests, else delegate) and the bounded env. core is NOT
gate-covered, so delegation warns loudly rather than false-greens.
- Lower CLASS_BUDGET_BANDS.changed ceiling 20min -> 13min so the script watchdog
fails a runaway local lane itself (exit 124, no restart) BEFORE the engine's
15-min kill restarts the whole task. A tightening, not a timeout-widening.
Guard test pins ceiling < 900_000ms.
- Raise scoped-affected worker fan-out 1 -> 4 (operator decision). Was 1 only
for OOM safety (FN-6854/FN-6874); the fan-out guard now bounds the set so the
hundreds-of-files OOM driver no longer reaches these workers. Heap stays
6144MB/worker (~4x6GB on the lane) — revisit if a RAM-constrained CI runner
OOMs. Trades FN-5048 worker-knob guidance for throughput, scoped to the
bounded affected lanes only.
Tests: test-changed 117/117, watchdog 15/15, eslint clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## What
A **test value audit**: a heuristic that scores every test file under
`packages/*/src/**/__tests__/**` and
`packages/dashboard/app/**/__tests__/**` by how much real regression
signal it has encoded over its git history, then ranks **slow AND
low-value** files first as deletion candidates. This is the **evidence
base** for a human/follow-up deletion decision — **the script deletes
nothing**.
Motivated by AGENTS.md "Do Not Add Slow Tests" (FN-5048) + the
quarantine deletion ratchet: we want data on which slow tests are also
low-signal so they can be cut without losing coverage.
## How it scores (HEURISTIC, not ground truth)
Single whole-history `git log --name-status` pass; per-commit
classification (renames followed backward):
- **Positive** — `fix(...)`/`fix:` with sibling source change (+3), fix
alone (+2), `## Symptom Verification` regression marker (+3, FN-5893),
test added-with-source (+2)/alone (+1), plain test+source co-change
(+1.5).
- **Negative** — subject/body churn keywords
`flake/flaky/deflake/quarantine/stabiliz/appease/timeout/retry` (−3),
test-only modify with no source (−1), quarantine-ledger membership
current+historical (−5).
`valueScore` = sum of weights. `deletionPriority = durationMs / (1 +
max(0, valueScore))` (+ small net-negative boost), joined with
`scripts/test-timings.json` so slow + low-value surfaces first.
Recommendation: `delete` (≤0) / `review` (≤3) / `keep` (>3). A
`safeDelete` flag marks files meeting the ratchet's churn/quarantine
bar.
## Deliverables
- `scripts/test-value-audit.mjs` — runner (git IO + report generation)
- `scripts/lib/test-value-audit-lib.mjs` — pure, unit-tested scoring
logic
- `scripts/__tests__/test-value-audit.test.mjs` — 15 synthetic-record
unit tests
- `docs/test-value-audit.json` + `docs/test-value-audit.md` — generated
artifacts (top 40 + methodology + honest caveats: heuristic limits,
git-follow/squash-merge blind spots, lying subjects, timing snapshot)
## Verification
- `node scripts/test-value-audit.mjs` runs end-to-end (~1s), 2051 files
analyzed, writes both artifacts.
- `node --test scripts/__tests__/test-value-audit.test.mjs` → 15/15
pass.
- `eslint` clean on all three new source files.
No changeset (scripts + docs only; `@runfusion/fusion` runtime
unaffected).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- stage-review-badge-begin -->
---
<a href="https://stagereview.app/Runfusion/Fusion/pull/1776">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://stagereview.app/assets/gh-open-in-stage-dark.svg">
<img src="https://stagereview.app/assets/gh-open-in-stage-light.svg"
alt="Open in Stage">
</picture>
</a>
<!-- stage-review-badge-end -->
Add scripts/test-value-audit.mjs + scripts/lib/test-value-audit-lib.mjs:
a heuristic that scores every test file under packages/*/src/**/__tests__/**
and packages/dashboard/app/**/__tests__/** from git history, classifying
commits as positive (fix+source co-change, Symptom Verification regressions,
added-with-source) vs negative (flake/quarantine/timeout churn, test-only
modifies, quarantine-ledger membership). Joins per-file durations from
scripts/test-timings.json so SLOW + LOW-VALUE files rank first as deletion
candidates. Emits docs/test-value-audit.json + docs/test-value-audit.md
(top 40 + methodology + caveats). Pure scoring logic is unit-tested with
synthetic commit records. The script never deletes tests — evidence only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Quarantine three test files consistently failing on the non-blocking
full-suite CI on main, per the AGENTS.md deletion-ratchet policy:
- engine self-healing-fn-5488-fast-path-regressions.test.ts (shard 1)
- engine in-review-merge-stall-deadlock-recovery.test.ts (shard 2)
- dashboard DevServerView.mobile.test.tsx (shard 4)
Each has a matching entry in scripts/lib/test-quarantine.json with
the failing CI run link and quarantinedAt date. Tests will be deleted
after 14 days unless rescued with a root-cause fix.
Update AGENTS.md, RELEASING.md, contributing.md with the structured
changeset format guide. Add .changeset/README.md template. Add changeset
for this change. Update distill-release-notes.mjs with final version.
Capture changeset entries before 'changeset version' deletes them,
distill deterministic notes post-version, and replace the version's
section in the root CHANGELOG with curated end-user notes.
Add scripts/lib/distill-release-notes.mjs with:
- distillDeterministic(): groups parsed changesets by category,
renders clean markdown release notes (Keep a Changelog headings)
- buildDistillationPrompt(): builds the context input for AI mode
- DISTILLATION_SYSTEM_PROMPT: system prompt for AI distillation
- Category display order: New → Fixed → Breaking → Security →
Performance → Internal; empty categories omitted
The AI mode (createFnAgent integration) is deferred to U4 where the
engine runtime is available. The deterministic mode is the fallback
contract: it always produces valid output regardless of model access.
14 tests covering category grouping, ordering, empty/edge cases,
prompt building, and legacy entry handling.
Remove the orphaned session-cross-tab quarantine ledger entry after confirming the test remains active and rescued.
- Document FN-6937's rescue proof and ledger/config lockstep expectation in testing guidance.
- Keep the dashboard Vitest quarantine list empty with a note explaining the stale ledger-only cleanup.
- Remove the session-cross-tab row from the flaky-test quarantine ledger.
Files changed:
docs/testing.md | 8 ++++++--
packages/dashboard/vitest.config.ts | 3 +++
scripts/lib/test-quarantine.json | 8 +-------
3 files changed, 10 insertions(+), 9 deletions(-)
Fusion-Task-Id: FN-6937
Fusion-Task-Lineage: 34a67bf3-2a7f-49ea-93ab-68fadc81f893
Resolve conflicts from main's analytics schema additions (plugin
activations, per-model token buckets) against the PR's contract-assertion
type column:
- db.ts: renumber behavioral-verification migration 124 -> 126, bump
SCHEMA_VERSION to 126 so it follows main's migrations 124/125
- core/roadmap tests: adopt main's SCHEMA_VERSION-constant assertions
instead of stale literal 124
- test-quarantine.json: keep all four quarantine entries from both sides
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Reformat shard-floor justification as an FNXC:TestInfrastructure comment
(project-standards: AGENTS.md FNXC_LOG convention).
- Clarify that the shard and dashboard-lane 15min floors are not coupled and
may diverge (maintainability: avoid implying an unenforced contract).
- Add a regression-guard test pinning shard.floor=15min and asserting a 525s
derived budget clamps up to the floor, so an accidental revert to the old
5min floor fails loudly (correctness + testing + project-standards).
The Full Suite (non-blocking) workflow has been red for 30+ runs on main.
Diagnosis: the @fusion/engine [1/2], [2/2] and @fusion/core [2/2] shard
slices were SIGKILLed at their watchdog budgets (405s/405s/338s), not because
they hang but because those budgets are too tight for current wall-clock.
Local baselines (this machine, all pass, exit 0):
- engine [2/2]: 145s wall / 309 files
- core [2/2]: 283s wall / 172 files (old budget was only 338s!)
deriveBudgetMs tightens the budget to expected*3.5 whenever the committed
scripts/test-timings.json is <30d old. The snapshot (2026-06-03) undercounts
the import- and real-git-subprocess overhead of these heavy slices, so the
'fresh' snapshot produced a too-tight, false-kill budget on slower CI runners
-- the exact failure mode the floor/ceiling band exists to prevent.
Fix (plan KTD-2): raise the shard band floor 5min -> 15min so the heaviest
slices can't be tightened into a false-kill, while a true hang is still bounded
far under the job's 60min ceiling. Mirrors the dashboard-lane heavy-lane floor.
Follow-up: refresh scripts/test-timings.json from a default-branch CI run.
Rescue the WorkflowNodeEditor quarantine by making duplicate merge seam detection reliable during initial canvas load.
- Derive fragment seam conflicts from the loaded workflow IR until React Flow nodes materialize.
- Treat IR merge nodes as merge seams so duplicate merge fragments are rejected consistently.
- Cover desktop and mobile duplicate merge fragment insertion paths and remove the test from quarantine.
- Document the FN-6744 rescue evidence and deletion-ratchet update.
Files changed:
docs/testing.md | 6 +++-
.../app/components/WorkflowNodeEditor.tsx | 11 +++++--
.../__tests__/WorkflowNodeEditor.test.tsx | 34 ++++++++++++++++------
.../app/components/workflow-flow-mapping.ts | 10 +++++++
packages/dashboard/vitest.config.ts | 8 +++--
scripts/lib/test-quarantine.json | 5 ----
6 files changed, 54 insertions(+), 20 deletions(-)
Fusion-Task-Id: FN-6744
Fusion-Task-Lineage: 0f5b167a-8efc-4458-ac9d-e719320b751f
Contain Command Center token totals so large formatted values stay inside their stat surfaces.
- Add shrink and wrap containment to stat and live metric value styles.
- Cover large comma-grouped token totals across Overview, Tokens, and Team surfaces.
- Guard the emitted CSS declarations and quarantine an unrelated WorkflowNodeEditor flake observed during verification.
Files changed:
.../components/command-center/CommandCenter.css | 10 +++++
.../__tests__/CommandCenter.test.tsx | 9 ++++
.../CommandCenter.token-validity.css.test.ts | 21 ++++++++++
.../command-center/areas/__tests__/areas.test.tsx | 48 ++++++++++++++++++----
packages/dashboard/vitest.config.ts | 5 +++
scripts/lib/test-quarantine.json | 5 +++
6 files changed, 89 insertions(+), 9 deletions(-)
Fusion-Task-Id: FN-6726
Fusion-Task-Lineage: 8a26ca5c-fdbe-4e68-a9d2-ae93340fe685
pnpm dev/local dashboard prebuild now rebuilds @fusion/core and @fusion/engine alongside the dashboard UI (was client-only), and startup warns loudly when built dist/ is older than src/. Prevents the FN-6638 class where landed engine fixes silently never run because the process loads stale dist.
Rescue the Paperclip CLI spawn coverage by making fake child events deterministic and re-enabling the suite.
- Route mintAgentApiKeyViaCli cases through the shared fake spawn helper so close and error events are emitted after listener registration.
- Document the async import/listener-order requirement that caused the quarantine.
- Remove paperclip-client.test.ts from the quarantine ledger and Vitest exclude list so the rescued tests run.
Files changed:
.../src/__tests__/paperclip-client.test.ts | 134 +++++++--------------
.../vitest.config.ts | 2 -
scripts/lib/test-quarantine.json | 5 -
3 files changed, 44 insertions(+), 97 deletions(-)
Fusion-Task-Id: FN-6636
Fusion-Task-Lineage: 79038ab3-0aa6-465e-9c6e-f6bc83fcd407
Restore the compound-engineering broad lane tests after the timeout could not be reproduced on the stabilized shared test infrastructure.
- Re-enable the compound-engineering sync and work-bridge tests by clearing their Vitest quarantine excludes.
- Remove the stale quarantine ledger entries now that the broad package lane stayed stable under verification.
Files changed:
plugins/fusion-plugin-compound-engineering/vitest.config.ts | 10 ++++------
scripts/lib/test-quarantine.json | 13 +------------
2 files changed, 5 insertions(+), 18 deletions(-)
Fusion-Task-Id: FN-6606
Fusion-Task-Lineage: f357c626-c181-44a7-a463-c53644566ca1
Normalize spawned Vitest processes to NODE_ENV=test so React/jsdom lanes do not inherit release-shell production mode. Split the compound-engineering plugin Vitest projects so Node-only setup and teardown only run in Node lanes, and quarantine the remaining broad-lane CE timeout flakes under the deletion ratchet.
Quarantine compound-engineering timeout flakes from the broad test workflow.
- Add the compound-engineering orchestrator and skill-wiring tests to the quarantine ledger.
- Exclude the quarantined tests from the compound-engineering node Vitest project without changing timeouts or retries.
- Document the quarantine requirement next to the Vitest excludes.
Files changed:
.../fusion-plugin-compound-engineering/vitest.config.ts | 15 ++++++++++++++-
scripts/lib/test-quarantine.json | 10 ++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
Fusion-Task-Id: FN-6587
Fusion-Task-Lineage: 5aeb0202-8330-4a93-bdab-675c81b9cb54
Rescues the QuickEntryBox suite by restoring jsdom globals instead of keeping it quarantined.
- Snapshot and restore QuickEntryBox viewport, visibility, matchMedia, and object URL descriptors after each test.
- Add a guard test that proves mutated jsdom globals return to their original descriptors.
- Remove the QuickEntryBox quarantine entries while documenting the rescue pattern.
Files changed:
docs/testing.md | 2 +
.../components/__tests__/QuickEntryBox.test.tsx | 50 ++++++++++++++++++++++
packages/dashboard/vitest.config.ts | 7 ++-
scripts/lib/test-quarantine.json | 5 ---
4 files changed, 55 insertions(+), 9 deletions(-)
Fusion-Task-Id: FN-6514
Fusion-Task-Lineage: 12f20ad8-19ee-4168-91a9-33193b1ab5f7
Rescue the same-day core and CLI flaky quarantines by fixing their fixture seams instead of appeasing timeouts.
- Make the core concurrent-write lock helper release synchronously inside its child process so package load cannot delay the transient lock release.
- Close real CLI TaskStore fixtures before removing temp roots and switch mock cleanup to non-hoisted unmocking.
- Remove both test files from package quarantine excludes and clear the quarantine ledger while documenting the rescue pattern.
Files changed:
docs/testing.md | 2 ++
packages/cli/src/__tests__/extension-task-tools.test.ts | 17 +++++++++++++----
packages/cli/vitest.config.ts | 4 +++-
.../core/src/__tests__/store-concurrent-writes.test.ts | 8 +++++++-
packages/core/vitest.config.ts | 4 +++-
scripts/lib/test-quarantine.json | 13 +------------
6 files changed, 29 insertions(+), 19 deletions(-)
Fusion-Task-Id: FN-6486
Fusion-Task-Lineage: c90358c4-0549-4c68-8d17-2e2a336433b5
