FN-8497: reduce merge gate wall time

Keep merge-gate coverage focused while running its independent test lanes concurrently.

- Limit PostgreSQL gate coverage to lifecycle and transactional-handoff canaries.
- Run engine and PostgreSQL gate lanes concurrently while preserving failure propagation.
- Enforce canary coverage policy and refresh velocity documentation and history.

Files changed:
 docs/test-velocity-baseline.md                     |  16 +--
 docs/testing.md                                    |   5 +-
 package.json                                       |   2 +-
 packages/core/package.json                         |   2 +-
 .../__tests__/engine-vitest-gate-policy.test.mjs   |  79 +++++++++++++-
 scripts/test-velocity-history.json                 | 115 +++++++++++++++++++++
 6 files changed, 204 insertions(+), 15 deletions(-)

Fusion-Task-Id: FN-8497
Fusion-Task-Lineage: 8777959c-6d8c-4686-a975-d91af2c169ea
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
gsxdsm
2026-07-22 15:47:24 -07:00
parent 55c0b7532c
commit 9002fca9de
6 changed files with 204 additions and 15 deletions

View File

@@ -5,7 +5,7 @@
## Latest baseline
- Cycle: **2026-W30**
- Captured at: **2026-07-22T21:58:46.970Z**
- Captured at: **2026-07-22T22:40:53.357Z**
- Timing snapshot: `scripts/test-timings.json` captured at **2026-06-27T05:41:42.568Z**
- Quarantine ledger: `scripts/lib/test-quarantine.json`
@@ -13,9 +13,9 @@
| Metric | Current | Delta vs previous |
|---|---:|---:|
| Merge gate wall-time (`pnpm test:gate`) | 29.3s | -15.9s |
| Boot smoke wall-time (`pnpm smoke:boot`) | 24.0s | +5.8s |
| Changed-only test wall-time (`pnpm test`) | 35.7s | -1.0s |
| Merge gate wall-time (`pnpm test:gate`) | 9.1s | -20.1s |
| Boot smoke wall-time (`pnpm smoke:boot`) | 18.5s | -5.6s |
| Changed-only test wall-time (`pnpm test`) | 14.9s | -20.7s |
| Quarantine / flake count | 2 | 0 |
| Deletion-due quarantines | 0 | n/a |
@@ -72,16 +72,16 @@
| Row | Captured at | Gate | Boot smoke | `pnpm test` | Quarantine count |
|---|---|---:|---:|---:|---:|
| Previous | 2026-07-22T20:52:05.508Z | 45.2s | 18.3s | 36.7s | 2 |
| Latest | 2026-07-22T21:58:46.970Z | 29.3s | 24.0s | 35.7s | 2 |
| Delta | — | -15.9s | +5.8s | -1.0s | 0 |
| Previous | 2026-07-22T21:58:46.970Z | 29.3s | 24.0s | 35.7s | 2 |
| Latest | 2026-07-22T22:40:53.357Z | 9.1s | 18.5s | 14.9s | 2 |
| Delta | — | -20.1s | -5.6s | -20.7s | 0 |
_Future weekly rows append to `scripts/test-velocity-history.json`; compare the latest row against the previous row before posting to #leads._
## Post to #leads
```text
FN-6612 weekly test velocity: gate 29.3s (-15.9s), boot smoke 24.0s (+5.8s), pnpm test 35.7s (-1.0s), quarantine ledger 2 (0). Slowest file: packages/dashboard/src/__tests__/insights-routes.test.ts at 26.5s. Deletion-due quarantines: 0.
FN-6612 weekly test velocity: gate 9.1s (-20.1s), boot smoke 18.5s (-5.6s), pnpm test 14.9s (-20.7s), quarantine ledger 2 (0). Slowest file: packages/dashboard/src/__tests__/insights-routes.test.ts at 26.5s. Deletion-due quarantines: 0.
```
## How to refresh

View File

@@ -6,10 +6,13 @@ This guide consolidates the detailed testing guidance moved from `AGENTS.md`.
## The merge gate
CI blocks PRs on exactly four checks (`.github/workflows/pr-checks.yml`): **Lint, Typecheck, Build, Gate**. The Gate job runs the boot smoke (`scripts/boot-smoke.mjs`: CLI `--help` + a real `fn serve` answering `GET /api/health`) and `pnpm test:gate` (the curated `engine-core` vitest project + the CI-shape test). Everything else — the 4-way shards, the engine slow tier, the dashboard inventory guard — runs NON-BLOCKING in `.github/workflows/full-suite.yml` on push to main.
CI blocks PRs on exactly four checks (`.github/workflows/pr-checks.yml`): **Lint, Typecheck, Build, Gate**. The Gate job runs the boot smoke (`scripts/boot-smoke.mjs`: CLI `--help` + a real `fn serve` answering `GET /api/health`) and `pnpm test:gate` (static process guards, curated `engine-core`, two PostgreSQL canaries, and the CI-shape test). Everything else — the 4-way shards, the engine slow tier, the dashboard inventory guard — runs NON-BLOCKING in `.github/workflows/full-suite.yml` on push to main.
Gate membership is the explicit allow-list in `packages/engine/vitest.config.ts` (`engine-core` project). Admission requires evidence of value (the test catches real regressions); tests never graduate in by default. A flaky gate test is evicted by deleting its allow-list line — the eviction PR does not need the flaky test to pass. The whole `engine-core` project must stay under ~60s wall-clock.
<!-- FNXC:MergeGatePerformance 2026-07-22-15:35: FN-8497 restores the 8–10s gate band after a 23-file PG integration lane added 17s of serialized database-template work. Keep only the lifecycle and transactional-handoff canaries blocking; the other former members must remain enabled in the non-blocking core test lane and the structural policy test enforces that containment. -->
**PostgreSQL gate policy:** `packages/core`'s `test:pg-gate` intentionally runs only `task-lifecycle-e2e.pg.test.ts` and `handoff-to-review-atomicity.pg.test.ts`, preserving real-backend lifecycle and atomic-handoff canaries. It runs concurrently with `engine-core` after the static guards; the root script waits for **both** lanes and propagates either failure before running CI-shape. Every other former PG gate member remains enabled and discovered by the non-blocking command `pnpm --filter @fusion/core test` (default config: `src/**/*.test.ts`, no PG quarantine exclusions). `scripts/__tests__/engine-vitest-gate-policy.test.mjs` pins the two canaries and fails if any removed member is deleted, undiscoverable, or hidden by the default lane's script/config.
<!-- FNXC:EngineTests 2026-07-08-03:00: FN-7667 decouples the engine-core gate's module graph from full-barrel growth so new feature modules don't silently inflate every gate fork's transform/import cost. -->
**Gate-safe `@fusion/core` barrel:** the `engine-core` project resolves `@fusion/core` to `packages/core/src/index.gate.ts` (a project-scoped `resolve.alias`, not the root map), not the full `packages/core/src/index.ts` barrel. `index.gate.ts` is a byte-for-byte copy of the full barrel minus the `export ... from` statements for modules added to the barrel after the last re-audit baseline — i.e. it re-exports everything the full barrel does except genuinely new, gate-irrelevant feature modules (diffed against the prior baseline commit's barrel, not hand-picked from what gate *test* files import — production modules under test pull in far more of the barrel transitively than their own imports suggest). `engine-default`/`engine-reliability`/`engine-slow` are unaffected and keep resolving the full barrel. `@fusion/engine` is untouched (no gate file imports it). When adding a new barrel module that no gate test needs, mirror the exclusion in `index.gate.ts` rather than letting gate wall-time grow — see the FNXC comment at the top of `index.gate.ts` and `packages/engine/vitest.config.ts`'s `engine-core` project for the audit procedure.

View File

@@ -21,7 +21,7 @@
"check:changesets": "node scripts/check-changeset-format.mjs",
"check:quarantine-ledger": "node scripts/check-quarantine-ledger.mjs",
"check:mock-completeness": "node scripts/check-mock-completeness.mjs",
"test:gate": "node scripts/check-no-nohup.mjs && node scripts/check-no-kill-4040.mjs && node scripts/check-no-getdatabase.mjs && node scripts/check-no-node-only-core-imports-in-dashboard.mjs && node scripts/check-pi-versions-pinned.mjs && node scripts/check-no-test-timeout-appeasement.mjs && node scripts/check-changeset-format.mjs && node scripts/check-mock-completeness.mjs && pnpm --filter @fusion/engine test:core && pnpm --filter @fusion/core test:pg-gate && pnpm --filter @runfusion/fusion test:ci-shape",
"test:gate": "node scripts/check-no-nohup.mjs && node scripts/check-no-kill-4040.mjs && node scripts/check-no-getdatabase.mjs && node scripts/check-no-node-only-core-imports-in-dashboard.mjs && node scripts/check-pi-versions-pinned.mjs && node scripts/check-no-test-timeout-appeasement.mjs && node scripts/check-changeset-format.mjs && node scripts/check-mock-completeness.mjs && sh -c 'pnpm --filter @fusion/engine test:core & engine_pid=$!; pnpm --filter @fusion/core test:pg-gate & pg_pid=$!; status=0; wait $engine_pid || status=1; wait $pg_pid || status=1; exit $status' && pnpm --filter @runfusion/fusion test:ci-shape",
"smoke:boot": "node scripts/boot-smoke.mjs",
"local": "node scripts/start-local.mjs",
"dev": "node scripts/dev-with-memory.mjs",

View File

@@ -43,7 +43,7 @@
"typecheck": "tsc --noEmit",
"test": "vitest run --silent=passed-only --reporter=dot",
"test:embedded-postgres": "vitest run src/__tests__/postgres/embedded-lifecycle.test.ts --silent=passed-only --reporter=dot",
"test:pg-gate": "vitest run --config vitest.pg.config.ts src/__tests__/postgres/handoff-to-review-atomicity.pg.test.ts src/__tests__/postgres/store-list.pg.test.ts src/__tests__/postgres/task-lifecycle-e2e.pg.test.ts src/__tests__/postgres/soft-delete-resurrection-FN-5233.pg.test.ts src/__tests__/postgres/agent-logs-and-monitor.pg.test.ts src/__tests__/postgres/todo-store.pg.test.ts src/__tests__/postgres/workflow-definitions.pg.test.ts src/__tests__/postgres/message-store.pg.test.ts src/__tests__/postgres/insight-store.pg.test.ts src/__tests__/postgres/insight-run-execution.pg.test.ts src/__tests__/postgres/research-store.pg.test.ts src/__tests__/postgres/mission-store.pg.test.ts src/__tests__/postgres/goal-store.pg.test.ts src/__tests__/postgres/artifacts-documents-evals.pg.test.ts src/__tests__/postgres/command-center-analytics.pg.test.ts src/__tests__/postgres/command-center-remaining-analytics.pg.test.ts src/__tests__/postgres/research-execution.pg.test.ts src/__tests__/postgres/async-store-events.pg.test.ts src/__tests__/postgres/signal-ingestion.pg.test.ts src/__tests__/postgres/mission-autopilot.pg.test.ts src/__tests__/postgres/workflow-create.pg.test.ts src/__tests__/postgres/monitor-trait-storm-guard.pg.test.ts src/__tests__/postgres/agent-wake-getagent.pg.test.ts --silent=passed-only --reporter=dot"
"test:pg-gate": "vitest run --config vitest.pg.config.ts src/__tests__/postgres/handoff-to-review-atomicity.pg.test.ts src/__tests__/postgres/task-lifecycle-e2e.pg.test.ts --silent=passed-only --reporter=dot"
},
"devDependencies": {
"@earendil-works/pi-coding-agent": "0.81.1",

View File

@@ -1,6 +1,6 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { readdirSync, readFileSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
@@ -58,10 +58,17 @@ test("engine-core remains an explicit allow-listed merge gate", () => {
const includeEntries = [...engineCoreBlock.matchAll(/"src\/__tests__\/[^"\n]+\.test\.ts"/g)].map((match) => match[0]);
assert.equal(new Set(includeEntries).size, includeEntries.length, "engine-core allow-list must not contain duplicates");
assert.ok(includeEntries.length >= 18, "engine-core allow-list must not be gutted to avoid the runtime abort");
/*
FNXC:MergeGatePerformance 2026-07-22-15:36:
FN-8497 exercises this policy after profiling the complete gate. The current
curated engine lane has 16 explicit files after the documented SQLite and
obsolete graph-runner retirements; guard its real floor instead of the stale
18-file count, while still requiring the replacement graph executor seam.
*/
assert.ok(includeEntries.length >= 16, "engine-core allow-list must not be gutted to avoid the runtime abort");
assert.ok(
includeEntries.includes('"src/__tests__/workflow-graph-task-runner.test.ts"'),
"engine-core must keep workflow graph gate coverage",
includeEntries.includes('"src/__tests__/workflow-graph-executor-parity.test.ts"'),
"engine-core must keep workflow graph executor gate coverage",
);
assert.ok(
includeEntries.includes('"src/__tests__/heartbeat-monitor.test.ts"'),
@@ -78,6 +85,70 @@ test("root and package gate scripts still propagate real Vitest failures", () =>
"vitest run --silent=passed-only --reporter=dot --project=engine-core",
);
assert.match(root.scripts?.["test:gate"] ?? "", /pnpm --filter @fusion\/engine test:core/);
assert.match(root.scripts?.["test:gate"] ?? "", /wait \$engine_pid \|\| status=1/);
assert.match(root.scripts?.["test:gate"] ?? "", /wait \$pg_pid \|\| status=1/);
assert.doesNotMatch(root.scripts?.["test:gate"] ?? "", /NODE_NO_WARNINGS/);
assert.doesNotMatch(root.scripts?.["test"] ?? "", /NODE_NO_WARNINGS/);
});
/*
FNXC:MergeGatePerformance 2026-07-22-15:35:
FN-8497 keeps only lifecycle and transactional-handoff PostgreSQL canaries in
`test:pg-gate`: 23 independent PG files each create/copy a real database, so
putting the whole integration inventory on every PR made the sequential merge
gate take 26–45 seconds. The other PG files must remain ordinary enabled core
tests; this structural guard prevents a future package-script/config change
from silently converting the speed fix into lost coverage.
*/
test("pg gate canaries remain a subset of the enabled non-blocking PG suite", () => {
const core = readJson("packages/core/package.json");
const coreConfig = read("packages/core/vitest.config.ts");
const pgDirectory = path.join(repoRoot, "packages/core/src/__tests__/postgres");
const discoveredPgFiles = new Set(
readdirSync(pgDirectory)
.filter((file) => file.endsWith(".pg.test.ts"))
.map((file) => `src/__tests__/postgres/${file}`),
);
const gateMembers = core.scripts?.["test:pg-gate"]?.match(/src\/__tests__\/postgres\/[^ ]+\.pg\.test\.ts/g) ?? [];
const expectedCanaries = [
"src/__tests__/postgres/handoff-to-review-atomicity.pg.test.ts",
"src/__tests__/postgres/task-lifecycle-e2e.pg.test.ts",
];
const formerGateMembers = [
...expectedCanaries,
"src/__tests__/postgres/store-list.pg.test.ts",
"src/__tests__/postgres/soft-delete-resurrection-FN-5233.pg.test.ts",
"src/__tests__/postgres/agent-logs-and-monitor.pg.test.ts",
"src/__tests__/postgres/todo-store.pg.test.ts",
"src/__tests__/postgres/workflow-definitions.pg.test.ts",
"src/__tests__/postgres/message-store.pg.test.ts",
"src/__tests__/postgres/insight-store.pg.test.ts",
"src/__tests__/postgres/insight-run-execution.pg.test.ts",
"src/__tests__/postgres/research-store.pg.test.ts",
"src/__tests__/postgres/mission-store.pg.test.ts",
"src/__tests__/postgres/goal-store.pg.test.ts",
"src/__tests__/postgres/artifacts-documents-evals.pg.test.ts",
"src/__tests__/postgres/command-center-analytics.pg.test.ts",
"src/__tests__/postgres/command-center-remaining-analytics.pg.test.ts",
"src/__tests__/postgres/research-execution.pg.test.ts",
"src/__tests__/postgres/async-store-events.pg.test.ts",
"src/__tests__/postgres/signal-ingestion.pg.test.ts",
"src/__tests__/postgres/mission-autopilot.pg.test.ts",
"src/__tests__/postgres/workflow-create.pg.test.ts",
"src/__tests__/postgres/monitor-trait-storm-guard.pg.test.ts",
"src/__tests__/postgres/agent-wake-getagent.pg.test.ts",
];
assert.deepEqual(gateMembers, expectedCanaries, "the PG gate must stay a narrow, explicit canary list");
assert.match(core.scripts?.test ?? "", /^vitest run\b/, "the non-blocking core lane must execute Vitest");
assert.doesNotMatch(core.scripts?.test ?? "", /\s(?:--exclude|--include)\b/, "the non-blocking core lane must not narrow discovery");
assert.match(coreConfig, /include:\s*\["src\/\*\*\/\*.test\.ts"\]/, "the default core config must discover PG tests");
assert.match(coreConfig, /const quarantinedCoreTests: string\[\] = \[\]/, "no PG test may be hidden by quarantine exclusion");
for (const file of formerGateMembers) {
assert.ok(discoveredPgFiles.has(file), `former PG gate member must remain discovered: ${file}`);
}
const removedFromGate = formerGateMembers.filter((file) => !gateMembers.includes(file));
assert.equal(removedFromGate.length, 21, "all non-canary former gate members must remain in the non-blocking lane");
assert.ok(removedFromGate.every((file) => discoveredPgFiles.has(file)), "removed PG members must remain discoverable");
});

View File

@@ -1930,6 +1930,121 @@
"Timing snapshot is 25 days old; verify slowest-file attribution before treating the table as the current culprit.",
"9 of the listed slowest files no longer exist at the recorded paths: `packages/dashboard/src/__tests__/insights-routes.test.ts`, `packages/engine/src/runtimes/__tests__/in-process-runtime.test.ts`, `packages/dashboard/src/__tests__/workflow-routes.test.ts`, `packages/core/src/__tests__/db.test.ts`, `packages/core/src/__tests__/mission-store.test.ts`, …. Refresh scripts/test-timings.json before choosing a slow-test rewrite."
]
},
{
"capturedAt": "2026-07-22T22:40:53.357Z",
"gateMs": 9144,
"bootSmokeMs": 18454,
"testMs": 14943,
"quarantineCount": 2,
"slowestTop20": [
{
"file": "packages/dashboard/src/__tests__/insights-routes.test.ts",
"ms": 26500,
"package": "@fusion/dashboard"
},
{
"file": "packages/engine/src/runtimes/__tests__/in-process-runtime.test.ts",
"ms": 24700,
"package": "@fusion/engine"
},
{
"file": "packages/dashboard/src/__tests__/workflow-routes.test.ts",
"ms": 22000,
"package": "@fusion/dashboard"
},
{
"file": "packages/core/src/__tests__/db.test.ts",
"ms": 21200,
"package": "@fusion/core"
},
{
"file": "packages/dashboard/app/components/__tests__/GitManagerModal.test.tsx",
"ms": 16900,
"package": "@fusion/dashboard"
},
{
"file": "packages/core/src/__tests__/mission-store.test.ts",
"ms": 16000,
"package": "@fusion/core"
},
{
"file": "packages/cli/src/__tests__/extension.test.ts",
"ms": 15700,
"package": "@runfusion/fusion"
},
{
"file": "packages/dashboard/app/components/__tests__/AgentPromptsManager.test.tsx",
"ms": 14800,
"package": "@fusion/dashboard"
},
{
"file": "packages/dashboard/app/components/__tests__/App.test.tsx",
"ms": 14600,
"package": "@fusion/dashboard"
},
{
"file": "packages/dashboard/app/components/__tests__/TaskDetailModal.inline-editing-and-integrations.test.tsx",
"ms": 14100,
"package": "@fusion/dashboard"
},
{
"file": "packages/dashboard/app/components/__tests__/TaskDetailModal.rendering.test.tsx",
"ms": 13700,
"package": "@fusion/dashboard"
},
{
"file": "packages/dashboard/src/__tests__/routes-auth.test.ts",
"ms": 13600,
"package": "@fusion/dashboard"
},
{
"file": "packages/core/src/__tests__/agent-store.test.ts",
"ms": 13400,
"package": "@fusion/core"
},
{
"file": "packages/engine/src/__tests__/workspace-merger-idempotency.test.ts",
"ms": 12700,
"package": "@fusion/engine"
},
{
"file": "packages/engine/src/__tests__/self-healing-workspace.test.ts",
"ms": 11800,
"package": "@fusion/engine"
},
{
"file": "packages/engine/src/__tests__/pr-response-run.test.ts",
"ms": 11600,
"package": "@fusion/engine"
},
{
"file": "packages/dashboard/app/components/__tests__/ListView.test.tsx",
"ms": 11300,
"package": "@fusion/dashboard"
},
{
"file": "plugins/fusion-plugin-compound-engineering/src/__tests__/sync.test.ts",
"ms": 11000,
"package": "@fusion-plugin-examples/compound-engineering"
},
{
"file": "packages/dashboard/app/components/__tests__/AgentDetailView.settings.test.tsx",
"ms": 10700,
"package": "@fusion/dashboard"
},
{
"file": "packages/dashboard/app/components/__tests__/SecretsView.test.tsx",
"ms": 10700,
"package": "@fusion/dashboard"
}
],
"measurementFailures": [],
"timingSnapshotCapturedAt": "2026-06-27T05:41:42.568Z",
"timingNotes": [
"Timing snapshot is 25 days old; verify slowest-file attribution before treating the table as the current culprit.",
"9 of the listed slowest files no longer exist at the recorded paths: `packages/dashboard/src/__tests__/insights-routes.test.ts`, `packages/engine/src/runtimes/__tests__/in-process-runtime.test.ts`, `packages/dashboard/src/__tests__/workflow-routes.test.ts`, `packages/core/src/__tests__/db.test.ts`, `packages/core/src/__tests__/mission-store.test.ts`, …. Refresh scripts/test-timings.json before choosing a slow-test rewrite."
]
}
]
}