From 845fad2aa40244a8a2b52784dc1122a6ff6f5ba8 Mon Sep 17 00:00:00 2001 From: gsxdsm Date: Wed, 8 Jul 2026 02:45:47 -0700 Subject: [PATCH] FN-7666: record weekly test-velocity regression finding (gate 7.7s -> 36.3s) Narrative: update the 2026-W28 test-velocity baseline report with the newly measured merge-gate/changed-test wall-time regression and document root-cause attribution. - Bump baseline cycle to 2026-W28 (captured 2026-07-08T09:30:50.606Z) - Update headline metrics table: gate 7.7s -> 36.3s (+28.6s), boot smoke 17.4s -> 25.3s (+7.9s), pnpm test 9.3s -> 43.4s (+34.0s), quarantine count unchanged at 1 - Add FN-7666 finding note attributing the regression to ~4,305 new re-exported source lines across 14 modules pulled through the @fusion/core / @fusion/engine barrels, inflating per-fork import/transform cost across the 18 curated engine-core gate files; confirms it is a genuine regression (not cache/env noise) via cold-vs-warm reruns; files follow-up FN-7667 for the barrel-decoupling fix - Update weekly history table (previous/latest/delta rows) and the #leads post text block - Append new row capturing this week's measurements to scripts/test-velocity-history.json Files changed: docs/test-velocity-baseline.md | 23 ++++---- scripts/test-velocity-history.json | 112 +++++++++++++++++++++++++++++++++++++ 2 files changed, 123 insertions(+), 12 deletions(-) Fusion-Task-Id: FN-7666 Fusion-Task-Lineage: 73ec627c-2fc6-4289-8e6e-3e839ed5c675 Co-authored-by: Fusion (runfusion.ai) --- docs/test-velocity-baseline.md | 23 +++--- scripts/test-velocity-history.json | 112 +++++++++++++++++++++++++++++ 2 files changed, 123 insertions(+), 12 deletions(-) diff --git a/docs/test-velocity-baseline.md b/docs/test-velocity-baseline.md index 70a0200b7d..8f47a7d3f5 100644 --- a/docs/test-velocity-baseline.md +++ b/docs/test-velocity-baseline.md @@ -4,8 +4,8 @@ ## Latest baseline -- Cycle: **2026-W27** -- Captured at: **2026-07-02T08:47:17.721Z** +- Cycle: **2026-W28** +- Captured at: **2026-07-08T09:30:50.606Z** - Timing snapshot: `scripts/test-timings.json` captured at **2026-06-27T05:41:42.568Z** - Quarantine ledger: `scripts/lib/test-quarantine.json` @@ -13,10 +13,10 @@ | Metric | Current | Delta vs previous | |---|---:|---:| -| Merge gate wall-time (`pnpm test:gate`) | 7.7s | +778ms | -| Boot smoke wall-time (`pnpm smoke:boot`) | 17.4s | -557ms | -| Changed-only test wall-time (`pnpm test`) | 9.3s | -4.4s | -| Quarantine / flake count | 1 | +1 | +| Merge gate wall-time (`pnpm test:gate`) | 36.3s | +28.6s | +| Boot smoke wall-time (`pnpm smoke:boot`) | 25.3s | +7.9s | +| Changed-only test wall-time (`pnpm test`) | 43.4s | +34.0s | +| Quarantine / flake count | 1 | 0 | | Deletion-due quarantines | 0 | n/a | ## Measurement failures @@ -26,8 +26,7 @@ ## Timing snapshot notes - No stale or missing timing metadata detected in the rendered slowest-file rows. - -## Slowest 20 test files +- **FN-7666 finding (2026-07-08):** the 2026-W28 merge-gate/changed-test wall-time jump (gate 7.7s → 36.3s, `pnpm test` 9.3s → 43.4s) is a **genuine transform/import regression, not environmental/cache noise** — confirmed via cold-vs-warm `engine-core` reruns (cold `real 22.0s` vs warm `real 20.6s`, only a ~6% delta, on an 88.7%-idle 28-core box). Bisection against the 2026-W27 baseline commit (`2d8f087a9`, gate=7.7s) found no `vitest.config.ts` pool/alias/include changes in `engine-core`; instead ~30 legitimate feature commits landed between 2026-07-02 and 2026-07-08 (planner-overseer/oversight, GitLab tracking+analytics, git-revert/AI-undo, provider registration/auth-storage split, bundled-plugin auto-install, Coding-Ideas workflow) that newly re-export ~4,305 source lines across 14 modules (largest: `packages/engine/src/task-revert.ts`, 1,652 lines) through the `packages/core/src/index.ts` / `packages/engine/src/index.ts` barrels. Because the `engine-core` vitest project resolves `@fusion/core`/`@fusion/engine` to those full barrels and runs each of its 18 curated gate files in its own `pool:"forks"` OS process, every fork independently re-imports and re-transforms the enlarged barrel even though none of the 18 gate tests exercise the new modules. None of the new modules pull in unusually heavy new third-party dependencies individually — the cost is the aggregate ~4.3K-line import surface × 18 forked processes. The proper fix (decoupling the 18 gate test files' imports from the full barrel, or splitting a lean gate-only entrypoint) is a broad cross-package test-infra rework outside FN-7666's Review-Level-1 scope; filed as follow-up **FN-7667** (depends on FN-7666). No `vitest.config.ts`/barrel edits were made for this finding — the next weekly capture should re-confirm this attribution and FN-7667's fix should return the gate to the ~7-10s band. | Rank | File | Package | Duration | |---:|---|---|---:| @@ -71,16 +70,16 @@ | Row | Captured at | Gate | Boot smoke | `pnpm test` | Quarantine count | |---|---|---:|---:|---:|---:| -| Previous | 2026-06-27T05:43:17.293Z | 6.9s | 18.0s | 13.7s | 0 | -| Latest | 2026-07-02T08:47:17.721Z | 7.7s | 17.4s | 9.3s | 1 | -| Delta | — | +778ms | -557ms | -4.4s | +1 | +| Previous | 2026-07-02T08:47:17.721Z | 7.7s | 17.4s | 9.3s | 1 | +| Latest | 2026-07-08T09:30:50.606Z | 36.3s | 25.3s | 43.4s | 1 | +| Delta | — | +28.6s | +7.9s | +34.0s | 0 | _Future weekly rows append to `scripts/test-velocity-history.json`; compare the latest row against the previous row before posting to #leads._ ## Post to #leads ```text -FN-6612 weekly test velocity: gate 7.7s (+778ms), boot smoke 17.4s (-557ms), pnpm test 9.3s (-4.4s), quarantine ledger 1 (+1). Slowest file: packages/dashboard/src/__tests__/insights-routes.test.ts at 26.5s. Deletion-due quarantines: 0. +FN-6612 weekly test velocity: gate 36.3s (+28.6s), boot smoke 25.3s (+7.9s), pnpm test 43.4s (+34.0s), quarantine ledger 1 (0). Slowest file: packages/dashboard/src/__tests__/insights-routes.test.ts at 26.5s. Deletion-due quarantines: 0. ``` ## How to refresh diff --git a/scripts/test-velocity-history.json b/scripts/test-velocity-history.json index f504667e3e..774f3fa693 100644 --- a/scripts/test-velocity-history.json +++ b/scripts/test-velocity-history.json @@ -1361,6 +1361,118 @@ "measurementFailures": [], "timingSnapshotCapturedAt": "2026-06-27T05:41:42.568Z", "timingNotes": [] + }, + { + "capturedAt": "2026-07-08T09:30:50.606Z", + "gateMs": 36301, + "bootSmokeMs": 25333, + "testMs": 43360, + "quarantineCount": 1, + "slowestTop20": [ + { + "file": "packages/dashboard/src/__tests__/insights-routes.test.ts", + "ms": 26500, + "package": "@fusion/dashboard" + }, + { + "file": "packages/engine/src/runtimes/__tests__/in-process-runtime.test.ts", + "ms": 24700, + "package": "@fusion/engine" + }, + { + "file": "packages/dashboard/src/__tests__/workflow-routes.test.ts", + "ms": 22000, + "package": "@fusion/dashboard" + }, + { + "file": "packages/core/src/__tests__/db.test.ts", + "ms": 21200, + "package": "@fusion/core" + }, + { + "file": "packages/dashboard/app/components/__tests__/GitManagerModal.test.tsx", + "ms": 16900, + "package": "@fusion/dashboard" + }, + { + "file": "packages/core/src/__tests__/mission-store.test.ts", + "ms": 16000, + "package": "@fusion/core" + }, + { + "file": "packages/cli/src/__tests__/extension.test.ts", + "ms": 15700, + "package": "@runfusion/fusion" + }, + { + "file": "packages/dashboard/app/components/__tests__/AgentPromptsManager.test.tsx", + "ms": 14800, + "package": "@fusion/dashboard" + }, + { + "file": "packages/dashboard/app/components/__tests__/App.test.tsx", + "ms": 14600, + "package": "@fusion/dashboard" + }, + { + "file": "packages/dashboard/app/components/__tests__/TaskDetailModal.inline-editing-and-integrations.test.tsx", + "ms": 14100, + "package": "@fusion/dashboard" + }, + { + "file": "packages/dashboard/app/components/__tests__/TaskDetailModal.rendering.test.tsx", + "ms": 13700, + "package": "@fusion/dashboard" + }, + { + "file": "packages/dashboard/src/__tests__/routes-auth.test.ts", + "ms": 13600, + "package": "@fusion/dashboard" + }, + { + "file": "packages/core/src/__tests__/agent-store.test.ts", + "ms": 13400, + "package": "@fusion/core" + }, + { + "file": "packages/engine/src/__tests__/workspace-merger-idempotency.test.ts", + "ms": 12700, + "package": "@fusion/engine" + }, + { + "file": "packages/engine/src/__tests__/self-healing-workspace.test.ts", + "ms": 11800, + "package": "@fusion/engine" + }, + { + "file": "packages/engine/src/__tests__/pr-response-run.test.ts", + "ms": 11600, + "package": "@fusion/engine" + }, + { + "file": "packages/dashboard/app/components/__tests__/ListView.test.tsx", + "ms": 11300, + "package": "@fusion/dashboard" + }, + { + "file": "plugins/fusion-plugin-compound-engineering/src/__tests__/sync.test.ts", + "ms": 11000, + "package": "@fusion-plugin-examples/compound-engineering" + }, + { + "file": "packages/dashboard/app/components/__tests__/AgentDetailView.settings.test.tsx", + "ms": 10700, + "package": "@fusion/dashboard" + }, + { + "file": "packages/dashboard/app/components/__tests__/SecretsView.test.tsx", + "ms": 10700, + "package": "@fusion/dashboard" + } + ], + "measurementFailures": [], + "timingSnapshotCapturedAt": "2026-06-27T05:41:42.568Z", + "timingNotes": [] } ] }