32 lines
7.6 KiB
Markdown
32 lines
7.6 KiB
Markdown
# Testing Suite Hardening Bugs
|
|
|
|
_Started: 2026-05-05_
|
|
|
|
This is the living bug log for the testing-suite hardening work described in
|
|
`docs/testing-suite-quality-prd.md`. Each entry should stay factual: what broke,
|
|
why it broke, how it was fixed, and what command verified the fix.
|
|
|
|
## Open
|
|
|
|
| ID | Area | Symptom | Root cause | Planned fix | Status |
|
|
|---|---|---|---|---|---|
|
|
| _None_ | | | | | |
|
|
|
|
## Fixed
|
|
|
|
| ID | Area | Symptom | Root cause | Fix | Verification |
|
|
|---|---|---|---|---|---|
|
|
| TSH-001 | Local test selection | `pnpm test` could under-test shared package edits. | Changed-package resolution selected direct changed workspaces but not dependent workspaces that import them. | `scripts/test-changed.mjs` now expands affected packages through the workspace reverse-dependency graph. | `node --test scripts/__tests__/test-changed.test.mjs` and `pnpm test:full` |
|
|
| TSH-002 | Local test cache | Cached package passes could hide dirty worktree edits. | Cache keys were based on tracked blob SHAs and did not account for modified or untracked working-tree content. | Dirty affected files now bypass package cache reuse so local edits are exercised before PR. | `node --test scripts/__tests__/test-changed.test.mjs` and `pnpm test:full` |
|
|
| TSH-003 | PR CI coverage | PR sharding omitted plugin packages and `@fusion/pi-llama-cpp`. | The CI shard package list was hard-coded instead of derived from the workspace graph. | `scripts/ci-test-shard.mjs` now derives shard candidates from workspace packages with test scripts. | `node --test scripts/__tests__/test-changed.test.mjs scripts/__tests__/test-governance.test.mjs` and `pnpm test:full` |
|
|
| TSH-004 | Test governance | Runtime package Vitest configs bypassed shared worker budgeting/isolation conventions. | Configs had drifted independently across Droid, Pi Claude, and Pi Llama packages. | The configs now share the same thread pool sizing and isolation defaults, with a governance test to keep them aligned. | `node --test scripts/__tests__/test-governance.test.mjs` and `pnpm test:full` |
|
|
| TSH-005 | Plugin changed-test targeting | Plugin edits still forced the entire suite after package-aware targeting was added. | `shouldForceFullSuite()` treated every `plugins/**` path as broad repo surface area. | Plugin workspace paths are now resolved like package paths, so plugin edits run the relevant plugin tests instead of automatically falling back to all tests. | `node --test scripts/__tests__/test-changed.test.mjs` |
|
|
| TSH-006 | Skipped-test inventory | The skipped-test inventory documented the wrong gate ownership for the new and legacy extension suites. | The maintained extension integration lane and legacy exhaustive gate were renamed during implementation, but the inventory text still reflected the earlier split. | `docs/skipped-test-inventory.md` now lists `FUSION_TEST_EXTENSION_INTEGRATION` for the maintained built-extension test and `FUSION_TEST_LEGACY_EXTENSION_INTEGRATION` for the legacy suite. | `git diff --check`, `node --test scripts/__tests__/test-governance.test.mjs`, and `pnpm test:full` |
|
|
| TSH-008 | Test isolation | `custom-providers.test.ts` wrote to the real `~/.fusion/settings.json` path during the first local full-suite run. | The test used `os.homedir()` directly instead of a fixture home directory. | `readCustomProviders()` now accepts an injectable home directory, and the test uses a temp HOME fixture that is removed after each case. | `pnpm --filter @fusion/engine test -- src/__tests__/custom-providers.test.ts` (ran the full engine lane: 108 files, 3339 tests) |
|
|
| TSH-007 | Dashboard runtime/noise | `pnpm test:full` passed but dashboard tests took about 10 minutes locally and emitted repeated SQLite experimental warnings plus git default-branch hints. | Dashboard tests mix real SQLite-backed route coverage, git-backed route fixtures, and jsdom UI suites in one package lane. Node emits `node:sqlite` experimental warnings globally, some git fixtures used plain `git init`, and dashboard tests used deprecated `environmentMatchGlobs`. | Dashboard tests now use explicit Vitest projects for `dashboard-app` and `dashboard-api`, package scripts expose `test:app`/`test:api`/`test:browser-smoke` lanes, the shared Vitest setup filters known SQLite experimental warnings, fixture repos initialize with `main`, and low-value agent CSS class-presence assertions were cut in favor of focused layout/token/no-inline-style contracts. | `pnpm --filter @fusion/dashboard test:app` (287 files, 7254 tests), `pnpm --filter @fusion/dashboard test:api` (132 files, 3493 tests), `pnpm --filter @fusion/core exec vitest run src/__tests__/central-db.test.ts --silent=passed-only --reporter=dot`, and `pnpm test:full` |
|
|
| TSH-009 | PR action minutes | PR test shards built the whole workspace before running tests, duplicating the separate build gate. | `pr-checks.yml` had a `pnpm build` step inside every test-shard job. | Removed the pre-test build from PR test shards; package tests that require artifacts build their own fixtures, while `build` remains an explicit PR workflow job. | `pnpm --filter @runfusion/fusion exec vitest run src/__tests__/ci-workflow.test.ts --silent=passed-only --reporter=dot` and `pnpm build` |
|
|
| TSH-010 | Dashboard lane scripts | `pnpm --filter @fusion/dashboard test:api` failed with "No test files found" after converting dashboard to Vitest projects. | Positional glob filters were evaluated against all projects instead of selecting the intended dashboard project. | Dashboard lane scripts now use `--project dashboard-app` and `--project dashboard-api`. | `pnpm --filter @fusion/dashboard test:api` and `pnpm --filter @fusion/dashboard exec vitest run --project dashboard-app app/__tests__/agent-css-classes.test.ts --silent=passed-only --reporter=dot` |
|
|
| TSH-011 | PR build coverage | While reducing PR shard minutes, the workflow temporarily had no standalone `pnpm build` job. | The old workflow only had lint, typecheck, and test shards, so removing the shard-local build also removed PR build coverage. | Added an explicit `build` job to `pr-checks.yml` and updated the workflow contract test to require it. | `pnpm --filter @runfusion/fusion exec vitest run src/__tests__/ci-workflow.test.ts --silent=passed-only --reporter=dot` and `pnpm build` |
|
|
| TSH-012 | Warning filtering | The first SQLite warning filter also installed a `process.on("warning")` listener that could duplicate unrelated warnings. | Node still prints non-filtered warnings by default, so manually writing them again made future warning output noisier. | The shared setup now only wraps `process.emitWarning` for the known SQLite experimental warning, and the dashboard noisy-output marker list no longer suppresses generic trace-warning guidance. | `pnpm --filter @fusion/core typecheck`, `pnpm --filter @fusion/core exec vitest run src/__tests__/central-db.test.ts --silent=passed-only --reporter=dot`, and `pnpm test:full` |
|
|
| TSH-013 | Dashboard default runtime | The default dashboard package test still took about 9 minutes after the noise cleanup. | `pnpm --filter @fusion/dashboard test` continued to run every app/jsdom and API/node file, including exhaustive modal/view permutations and broad route matrices intended for deeper sweeps. | Added curated `dashboard-app-quality` and `dashboard-api-quality` Vitest projects for the default package gate, kept exhaustive coverage behind `test:deep`, `test:app`, and `test:api`, and documented when to run each lane. | `/usr/bin/time -p pnpm --filter @fusion/dashboard test` (148 files, 3642 tests, `real 91.73`), `/usr/bin/time -p pnpm --filter @fusion/dashboard test:deep` (419 files, 10747 tests, `real 335.47`), and `/usr/bin/time -p pnpm test:full` (`real 308.34`) |
|