fusion/docs/bugs/testing-suite-hardening.md

Testing Suite Hardening Bugs

Started: 2026-05-05

This is the living bug log for the testing-suite hardening work described in docs/testing-suite-quality-prd.md. Each entry should stay factual: what broke, why it broke, how it was fixed, and what command verified the fix.

Open

ID	Area	Symptom	Root cause	Planned fix	Status
None

Fixed

ID	Area	Symptom	Root cause	Fix	Verification
TSH-001	Local test selection	pnpm test could under-test shared package edits.	Changed-package resolution selected direct changed workspaces but not dependent workspaces that import them.	scripts/test-changed.mjs now expands affected packages through the workspace reverse-dependency graph.	node --test scripts/__tests__/test-changed.test.mjs and pnpm test:full
TSH-002	Local test cache	Cached package passes could hide dirty worktree edits.	Cache keys were based on tracked blob SHAs and did not account for modified or untracked working-tree content.	Dirty affected files now bypass package cache reuse so local edits are exercised before PR.	node --test scripts/__tests__/test-changed.test.mjs and pnpm test:full
TSH-003	PR CI coverage	PR sharding omitted plugin packages and @fusion/pi-llama-cpp.	The CI shard package list was hard-coded instead of derived from the workspace graph.	scripts/ci-test-shard.mjs now derives shard candidates from workspace packages with test scripts.	node --test scripts/__tests__/test-changed.test.mjs scripts/__tests__/test-governance.test.mjs and pnpm test:full
TSH-004	Test governance	Runtime package Vitest configs bypassed shared worker budgeting/isolation conventions.	Configs had drifted independently across Droid, Pi Claude, and Pi Llama packages.	The configs now share the same thread pool sizing and isolation defaults, with a governance test to keep them aligned.	node --test scripts/__tests__/test-governance.test.mjs and pnpm test:full
TSH-005	Plugin changed-test targeting	Plugin edits still forced the entire suite after package-aware targeting was added.	shouldForceFullSuite() treated every plugins/** path as broad repo surface area.	Plugin workspace paths are now resolved like package paths, so plugin edits run the relevant plugin tests instead of automatically falling back to all tests.	node --test scripts/__tests__/test-changed.test.mjs
TSH-006	Skipped-test inventory	The skipped-test inventory documented the wrong gate ownership for the new and legacy extension suites.	The maintained extension integration lane and legacy exhaustive gate were renamed during implementation, but the inventory text still reflected the earlier split.	docs/skipped-test-inventory.md now lists FUSION_TEST_EXTENSION_INTEGRATION for the maintained built-extension test and FUSION_TEST_LEGACY_EXTENSION_INTEGRATION for the legacy suite.	git diff --check, node --test scripts/__tests__/test-governance.test.mjs, and pnpm test:full
TSH-008	Test isolation	custom-providers.test.ts wrote to the real ~/.fusion/settings.json path during the first local full-suite run.	The test used os.homedir() directly instead of a fixture home directory.	readCustomProviders() now accepts an injectable home directory, and the test uses a temp HOME fixture that is removed after each case.	pnpm --filter @fusion/engine test -- src/__tests__/custom-providers.test.ts (ran the full engine lane: 108 files, 3339 tests)
TSH-007	Dashboard runtime/noise	pnpm test:full passed but dashboard tests took about 10 minutes locally and emitted repeated SQLite experimental warnings plus git default-branch hints.	Dashboard tests mix real SQLite-backed route coverage, git-backed route fixtures, and jsdom UI suites in one package lane. Node emits node:sqlite experimental warnings globally, some git fixtures used plain git init, and dashboard tests used deprecated environmentMatchGlobs.	Dashboard tests now use explicit Vitest projects for dashboard-app and dashboard-api, package scripts expose test:app/test:api/test:browser-smoke lanes, the shared Vitest setup filters known SQLite experimental warnings, fixture repos initialize with main, and low-value agent CSS class-presence assertions were cut in favor of focused layout/token/no-inline-style contracts.	pnpm --filter @fusion/dashboard test:app (287 files, 7254 tests), pnpm --filter @fusion/dashboard test:api (132 files, 3493 tests), pnpm --filter @fusion/core exec vitest run src/__tests__/central-db.test.ts --silent=passed-only --reporter=dot, and pnpm test:full
TSH-009	PR action minutes	PR test shards built the whole workspace before running tests, duplicating the separate build gate.	pr-checks.yml had a pnpm build step inside every test-shard job.	Removed the pre-test build from PR test shards; package tests that require artifacts build their own fixtures, while build remains an explicit PR workflow job.	pnpm --filter @runfusion/fusion exec vitest run src/__tests__/ci-workflow.test.ts --silent=passed-only --reporter=dot and pnpm build
TSH-010	Dashboard lane scripts	pnpm --filter @fusion/dashboard test:api failed with "No test files found" after converting dashboard to Vitest projects.	Positional glob filters were evaluated against all projects instead of selecting the intended dashboard project.	Dashboard lane scripts now use --project dashboard-app and --project dashboard-api.	pnpm --filter @fusion/dashboard test:api and pnpm --filter @fusion/dashboard exec vitest run --project dashboard-app app/__tests__/agent-css-classes.test.ts --silent=passed-only --reporter=dot
TSH-011	PR build coverage	While reducing PR shard minutes, the workflow temporarily had no standalone pnpm build job.	The old workflow only had lint, typecheck, and test shards, so removing the shard-local build also removed PR build coverage.	Added an explicit build job to pr-checks.yml and updated the workflow contract test to require it.	pnpm --filter @runfusion/fusion exec vitest run src/__tests__/ci-workflow.test.ts --silent=passed-only --reporter=dot and pnpm build
TSH-012	Warning filtering	The first SQLite warning filter also installed a process.on("warning") listener that could duplicate unrelated warnings.	Node still prints non-filtered warnings by default, so manually writing them again made future warning output noisier.	The shared setup now only wraps process.emitWarning for the known SQLite experimental warning, and the dashboard noisy-output marker list no longer suppresses generic trace-warning guidance.	pnpm --filter @fusion/core typecheck, pnpm --filter @fusion/core exec vitest run src/__tests__/central-db.test.ts --silent=passed-only --reporter=dot, and pnpm test:full
TSH-013	Dashboard default runtime	The default dashboard package test still took about 9 minutes after the noise cleanup.	pnpm --filter @fusion/dashboard test continued to run every app/jsdom and API/node file, including exhaustive modal/view permutations and broad route matrices intended for deeper sweeps.	Added curated dashboard-app-quality and dashboard-api-quality Vitest projects for the default package gate, kept exhaustive coverage behind test:deep, test:app, and test:api, and documented when to run each lane.	/usr/bin/time -p pnpm --filter @fusion/dashboard test (148 files, 3642 tests, real 91.73), /usr/bin/time -p pnpm --filter @fusion/dashboard test:deep (419 files, 10747 tests, real 335.47), and /usr/bin/time -p pnpm test:full (real 308.34)
TSH-014	Core temp-dir cleanup	kb-db-test-* directories could leak after db.test.ts runs and trip isolation checks.	afterEach(async) and module-level afterAll(async) both cleared createdTmpDirs before async rm(...) completed, so hook timing races could hide leftovers from final teardown during worker shutdown.	Coordinated cleanup bookkeeping by deleting set entries only after per-dir removal, kept afterEach as async best-effort cleanup, and switched the final afterAll fallback to defensive synchronous rmSync(..., { recursive: true, force: true }).	pnpm --filter @fusion/core test -- src/__tests__/db.test.ts and node scripts/check-test-isolation.mjs --before && pnpm --filter @fusion/core test && node scripts/check-test-isolation.mjs
TSH-015	PR lint workflow contract	PR lint spent extra CI minutes setting up Bun and running pnpm build before lint.	pr-checks.yml drifted from the CI pattern where lint should be the fastest install+lint gate and build should be enforced by the separate build job.	Removed Bun setup and pre-lint build from the PR lint job, and strengthened workflow contract tests to require pnpm install --frozen-lockfile + pnpm lint in lint while asserting Bun/build remain in the PR build job.	pnpm --filter @runfusion/fusion exec vitest run src/__tests__/ci-workflow.test.ts --silent=passed-only --reporter=dot, pnpm lint, pnpm test, pnpm typecheck, and pnpm build