- 427 orphaned dashboard test files ran in NO gate; 395 now gated via self-maintaining backfill lanes (glob minus curated minus skip-list), 31 pre-existing failures + build-output skip-listed with reasons - settings -t name-filter lanes replaced by one unfiltered lane (describe blocks can no longer fall through filters) - scripts/check-test-inventory.mjs: --capture/--diff superset harness + --dashboard-curated completeness guard - pr-checks.yml: engine-slow CI gate (non-empty assertion) + inventory guard job - docs/testing.md: guard, skip-list policy, harness usage
168 lines
9.5 KiB
Markdown
168 lines
9.5 KiB
Markdown
# Testing Guide
|
|
|
|
[← Docs index](./README.md)
|
|
|
|
This guide consolidates the detailed testing guidance moved from `AGENTS.md`.
|
|
|
|
## Required workspace gates
|
|
|
|
Tests are required. Typechecks and manual verification are not substitutes for assertions.
|
|
|
|
Use the narrowest command that exercises the behavior you changed, then broaden before reporting completion.
|
|
|
|
```bash
|
|
pnpm test # changed-only workspace tests; falls back to full gate in safety contexts
|
|
pnpm test:full # full workspace quality gate
|
|
pnpm lint # lint all packages
|
|
pnpm build # build workspace packages (excludes desktop/mobile)
|
|
pnpm verify:workspace # canonical pre-merge gate: lint -> test:full -> build
|
|
```
|
|
|
|
`pnpm test:full` runs each package's default test script with capped worker fanout (`FUSION_TEST_TOTAL_WORKERS=4 FUSION_TEST_CONCURRENCY=2 pnpm -r --workspace-concurrency=2 test`). Do not casually raise worker counts; dashboard/jsdom and integration-heavy packages destabilize when oversubscribed. Use `VITEST_MAX_WORKERS=<n>` only for targeted package-level investigation.
|
|
|
|
## Fresh-worktree dist bootstrap
|
|
|
|
`pnpm test` auto-runs `scripts/ensure-test-artifacts.mjs` to rebuild missing/stale dist artifacts. Dashboard and `dependency-graph` package lanes auto-bootstrap too. If you hit opaque `Failed to resolve import "./cli-spawn.js"` (or similar), treat it as bootstrap regression against FN-4605 — don't work around with a manual `pnpm build`.
|
|
|
|
## Dashboard Test Lanes
|
|
|
|
```bash
|
|
pnpm --filter @fusion/dashboard test # curated app/API quality gate (default)
|
|
pnpm --filter @fusion/dashboard test:deep # exhaustive app + API suite
|
|
pnpm --filter @fusion/dashboard test:app # exhaustive React/jsdom
|
|
pnpm --filter @fusion/dashboard test:api # exhaustive Node API/server
|
|
pnpm --filter @fusion/dashboard test:browser-smoke # local browser CSS/layout smoke
|
|
pnpm --filter @fusion/dashboard test:build # built client output contract
|
|
```
|
|
|
|
Run `test:deep` when changing broad dashboard architecture, shared modal/view infrastructure, or route registration. Run `test:browser-smoke` for layout/responsive/navigation/modal/CSS changes. Run `test:build` for Vite output, lazy-loading, chunking, or client-dist changes.
|
|
|
|
New test files under `app/**` or `src/**` are picked up automatically by the
|
|
**backfill lanes** (`dashboard-app-quality-backfill` / `dashboard-api-quality-backfill`),
|
|
which include the broad globs and exclude only the files an explicit curated lane
|
|
already runs plus the skip-list. You do not need to register a new file by hand for
|
|
it to run — the curated-gate hole that silently skipped unenumerated files is closed
|
|
(see "Curated-gate completeness" below). Add a file to a curated `qualityApp*`/`qualityApi`
|
|
list only when you want it in a specific fast lane rather than the backfill catch-all.
|
|
|
|
## Curated-gate completeness and the skip-list
|
|
|
|
The dashboard quality gate is a chain of curated lanes plus two backfill lanes.
|
|
Together they must execute **every** `*.test.{ts,tsx}` under `packages/dashboard/app`
|
|
and `packages/dashboard/src`, or the file must be on the reviewed skip-list. This is
|
|
enforced by a guard (CI job `Dashboard curated-gate guard` in `pr-checks.yml`):
|
|
|
|
```bash
|
|
node scripts/check-test-inventory.mjs --dashboard-curated
|
|
```
|
|
|
|
It fails when a dashboard test file is neither executed by a quality project nor
|
|
skip-listed. The skip-list lives at `scripts/lib/dashboard-curated-skiplist.json`;
|
|
every entry needs a non-empty `reason` (empty reasons are rejected). Skip-list policy:
|
|
|
|
- A file goes on the skip-list only when it genuinely cannot be gated yet — today
|
|
that is pre-existing-failing orphans (tests that were never executed in CI and
|
|
fail in isolation) and `build-output.test.ts` (runs standalone via `test:build`
|
|
after a Vite build). Each carries a one-line reason.
|
|
- To remove a file from the skip-list: fix the test, confirm it passes under its
|
|
project, delete the skip-list entry. The backfill lane then executes it.
|
|
- The skip-list is shared verbatim with `vitest.config.ts`, which excludes the same
|
|
globs from the backfill projects — one source of truth.
|
|
|
|
## Test-inventory harness
|
|
|
|
`scripts/check-test-inventory.mjs` is the standard coverage-superset verification
|
|
step. Node stdlib only.
|
|
|
|
```bash
|
|
# Snapshot the executed-test inventory (per package/project, normalized test ids).
|
|
node scripts/check-test-inventory.mjs --capture before.json
|
|
# ... make a change ...
|
|
node scripts/check-test-inventory.mjs --capture after.json
|
|
# Fail (exit 1) if any test id present in `before` is missing from `after`.
|
|
node scripts/check-test-inventory.mjs --diff before.json after.json
|
|
```
|
|
|
|
The capture spec (which packages/projects to enumerate) lives in
|
|
`scripts/lib/test-inventory-spec.json`. The diff lists the exact missing test ids;
|
|
a renamed file shows up as a remove (old path) + add (new path), so the rename is
|
|
reviewable. New test ids never fail the diff.
|
|
|
|
## Engine slow tier (CI gate)
|
|
|
|
The `engine-slow` vitest project (`packages/engine/src/**/*.slow.test.ts`) holds the
|
|
long real-git suites. It runs locally via `pnpm --filter @fusion/engine test:slow` and
|
|
in CI via the `Engine slow tier` job in `pr-checks.yml`, which uses
|
|
`scripts/assert-engine-slow-nonempty.mjs` to **fail if zero tests executed** (so a glob
|
|
or config drift that silently empties the tier breaks CI instead of passing vacuously).
|
|
The CI job uses `fetch-depth: 0` because these tests run real git operations.
|
|
|
|
## Targeted commands
|
|
|
|
```bash
|
|
pnpm --filter @fusion/core test
|
|
pnpm --filter @fusion/engine test
|
|
pnpm --filter @runfusion/fusion test
|
|
pnpm test:scripts
|
|
node --test scripts/__tests__/*.test.mjs
|
|
```
|
|
|
|
For a single Vitest file, use package-local `exec vitest`:
|
|
|
|
```bash
|
|
pnpm --filter @fusion/core exec vitest run src/__tests__/central-db.test.ts --silent=passed-only --reporter=dot
|
|
```
|
|
|
|
## Engine test helper convention
|
|
|
|
`packages/engine/src/__tests__/executor-test-helpers.ts` defaults both `isUsableTaskWorktree` to `true` and `classifyTaskWorktree` to `{ ok: true }` via a helper-level `worktree-pool` mock. To test failure paths, override with `vi.spyOn(worktreePool, "classifyTaskWorktree").mockResolvedValueOnce({ ok: false, classification: "unregistered", reason: "..." })` (or `isUsableTaskWorktree` for legacy call sites). Production liveness assertions in `executor.ts` are unchanged.
|
|
|
|
## Before reporting done
|
|
|
|
- Code changes: affected package tests + any directly relevant browser/build lane.
|
|
- Cross-package, shared test infrastructure, or CI changes: `pnpm test:full`.
|
|
- Production/bundling-sensitive changes: `pnpm build`.
|
|
- Substantial work: `pnpm verify:workspace`.
|
|
- If you skip a relevant lane, say why.
|
|
|
|
## Test file organization
|
|
|
|
Test for `src/foo.ts` → `src/__tests__/foo.test.ts`. Test for `app/components/Bar.tsx` → `app/components/__tests__/Bar.test.tsx`. `__tests__/` is the standard.
|
|
|
|
## What NOT to write
|
|
|
|
Tests should cover behavior a user could notice break, not implementation shape. Don't write:
|
|
|
|
- **CSS-class permutation tests** — use one `it.each` for the boolean matrix, not one `it` per combination.
|
|
- **Field-presence tests** when a payload-roundtrip test already exercises the same field.
|
|
- **React.memo tautologies** — testing `React.memo` tests React, not us. Test custom comparators directly, one case.
|
|
- **Mock-the-world wiring tests** — if a test mocks 8+ deps just to render a component, shim children with `() => null` or delete and rely on an integration test one level up.
|
|
- **Structural CSS assertions** — "tab uses .class-name not inline style". Consolidate into one aggregate layout-contract test per component.
|
|
|
|
Prefer `it.each` over copy-pasted `it()` blocks. When trimming, keep: first case + opposite case + any precedence/override case.
|
|
|
|
## What TO keep unconditionally
|
|
|
|
- Tests linked to an FN-ticket in describe/it names — these guard real regressions.
|
|
- Integration tests exercising real SQLite, real worker pool, or spawned processes.
|
|
- Lean core/engine unit tests with low mock burden.
|
|
|
|
## Standing Rule: Do Not Add Slow Tests (FN-5048)
|
|
|
|
- Default new tests to narrow seams, in-memory fakes, shared harnesses, and targeted assertions.
|
|
- For bug-fix regressions, also follow `AGENTS.md` → **Standing Rule: Fix the Invariant, Not the Repro (FN-5893)** so coverage proves the invariant across known surfaces, not just one repro.
|
|
- Prefer fake timers over real polling/time waits (FN-2707 pattern: advance timers inside `act(...)`, restore with `afterEach(() => vi.useRealTimers())`).
|
|
- Do **not** mask slowness by raising worker/concurrency knobs (`FUSION_TEST_TOTAL_WORKERS`, `FUSION_TEST_CONCURRENCY`, `VITEST_MAX_WORKERS`, workspace concurrency settings).
|
|
- Do **not** add net-new real-network calls, real-`setTimeout` polling loops, or mock-the-world component shells when a narrower seam exists.
|
|
- Use the canonical taxonomy in **What NOT to write** and **What TO keep unconditionally** when deciding trim vs keep.
|
|
- See `docs/test-speed-audit-FN-5048.md` for the measured baseline offender list and optimization priorities.
|
|
|
|
### Surface Enumeration checklist
|
|
|
|
Copy this checklist into a bug-fix task's `## Surface Enumeration` section and make the regression tests prove the invariant across every checked surface. See `AGENTS.md` → **Standing Rule: Fix the Invariant, Not the Repro (FN-5893)** for the enforced planning/review contract.
|
|
|
|
- [ ] Providers / bridges / execution paths touched by the invariant
|
|
- [ ] Desktop + mobile breakpoints / platforms that exercise the behavior
|
|
- [ ] Empty / undefined / duplicate / populated data states
|
|
- [ ] Shared hooks / components / modules / helpers reusing the logic
|