FN-7666: record weekly test-velocity regression finding (gate 7.7s -> 36.3s)
Narrative: update the 2026-W28 test-velocity baseline report with the newly measured merge-gate/changed-test wall-time regression and document root-cause attribution. - Bump baseline cycle to 2026-W28 (captured 2026-07-08T09:30:50.606Z) - Update headline metrics table: gate 7.7s -> 36.3s (+28.6s), boot smoke 17.4s -> 25.3s (+7.9s), pnpm test 9.3s -> 43.4s (+34.0s), quarantine count unchanged at 1 - Add FN-7666 finding note attributing the regression to ~4,305 new re-exported source lines across 14 modules pulled through the @fusion/core / @fusion/engine barrels, inflating per-fork import/transform cost across the 18 curated engine-core gate files; confirms it is a genuine regression (not cache/env noise) via cold-vs-warm reruns; files follow-up FN-7667 for the barrel-decoupling fix - Update weekly history table (previous/latest/delta rows) and the #leads post text block - Append new row capturing this week's measurements to scripts/test-velocity-history.json Files changed: docs/test-velocity-baseline.md | 23 ++++---- scripts/test-velocity-history.json | 112 +++++++++++++++++++++++++++++++++++++ 2 files changed, 123 insertions(+), 12 deletions(-) Fusion-Task-Id: FN-7666 Fusion-Task-Lineage: 73ec627c-2fc6-4289-8e6e-3e839ed5c675 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
gsxdsm
@@ -4,8 +4,8 @@
|
||||
|
||||
## Latest baseline
|
||||
|
||||
- Cycle: **2026-W27**
|
||||
- Captured at: **2026-07-02T08:47:17.721Z**
|
||||
- Cycle: **2026-W28**
|
||||
- Captured at: **2026-07-08T09:30:50.606Z**
|
||||
- Timing snapshot: `scripts/test-timings.json` captured at **2026-06-27T05:41:42.568Z**
|
||||
- Quarantine ledger: `scripts/lib/test-quarantine.json`
|
||||
|
||||
@@ -13,10 +13,10 @@
|
||||
|
||||
| Metric | Current | Delta vs previous |
|
||||
|---|---:|---:|
|
||||
| Merge gate wall-time (`pnpm test:gate`) | 7.7s | +778ms |
|
||||
| Boot smoke wall-time (`pnpm smoke:boot`) | 17.4s | -557ms |
|
||||
| Changed-only test wall-time (`pnpm test`) | 9.3s | -4.4s |
|
||||
| Quarantine / flake count | 1 | +1 |
|
||||
| Merge gate wall-time (`pnpm test:gate`) | 36.3s | +28.6s |
|
||||
| Boot smoke wall-time (`pnpm smoke:boot`) | 25.3s | +7.9s |
|
||||
| Changed-only test wall-time (`pnpm test`) | 43.4s | +34.0s |
|
||||
| Quarantine / flake count | 1 | 0 |
|
||||
| Deletion-due quarantines | 0 | n/a |
|
||||
|
||||
## Measurement failures
|
||||
@@ -26,8 +26,7 @@
|
||||
## Timing snapshot notes
|
||||
|
||||
- No stale or missing timing metadata detected in the rendered slowest-file rows.
|
||||
|
||||
## Slowest 20 test files
|
||||
- **FN-7666 finding (2026-07-08):** the 2026-W28 merge-gate/changed-test wall-time jump (gate 7.7s → 36.3s, `pnpm test` 9.3s → 43.4s) is a **genuine transform/import regression, not environmental/cache noise** — confirmed via cold-vs-warm `engine-core` reruns (cold `real 22.0s` vs warm `real 20.6s`, only a ~6% delta, on an 88.7%-idle 28-core box). Bisection against the 2026-W27 baseline commit (`2d8f087a9`, gate=7.7s) found no `vitest.config.ts` pool/alias/include changes in `engine-core`; instead ~30 legitimate feature commits landed between 2026-07-02 and 2026-07-08 (planner-overseer/oversight, GitLab tracking+analytics, git-revert/AI-undo, provider registration/auth-storage split, bundled-plugin auto-install, Coding-Ideas workflow) that newly re-export ~4,305 source lines across 14 modules (largest: `packages/engine/src/task-revert.ts`, 1,652 lines) through the `packages/core/src/index.ts` / `packages/engine/src/index.ts` barrels. Because the `engine-core` vitest project resolves `@fusion/core`/`@fusion/engine` to those full barrels and runs each of its 18 curated gate files in its own `pool:"forks"` OS process, every fork independently re-imports and re-transforms the enlarged barrel even though none of the 18 gate tests exercise the new modules. None of the new modules pull in unusually heavy new third-party dependencies individually — the cost is the aggregate ~4.3K-line import surface × 18 forked processes. The proper fix (decoupling the 18 gate test files' imports from the full barrel, or splitting a lean gate-only entrypoint) is a broad cross-package test-infra rework outside FN-7666's Review-Level-1 scope; filed as follow-up **FN-7667** (depends on FN-7666). No `vitest.config.ts`/barrel edits were made for this finding — the next weekly capture should re-confirm this attribution and FN-7667's fix should return the gate to the ~7-10s band.
|
||||
|
||||
| Rank | File | Package | Duration |
|
||||
|---:|---|---|---:|
|
||||
@@ -71,16 +70,16 @@
|
||||
|
||||
| Row | Captured at | Gate | Boot smoke | `pnpm test` | Quarantine count |
|
||||
|---|---|---:|---:|---:|---:|
|
||||
| Previous | 2026-06-27T05:43:17.293Z | 6.9s | 18.0s | 13.7s | 0 |
|
||||
| Latest | 2026-07-02T08:47:17.721Z | 7.7s | 17.4s | 9.3s | 1 |
|
||||
| Delta | — | +778ms | -557ms | -4.4s | +1 |
|
||||
| Previous | 2026-07-02T08:47:17.721Z | 7.7s | 17.4s | 9.3s | 1 |
|
||||
| Latest | 2026-07-08T09:30:50.606Z | 36.3s | 25.3s | 43.4s | 1 |
|
||||
| Delta | — | +28.6s | +7.9s | +34.0s | 0 |
|
||||
|
||||
_Future weekly rows append to `scripts/test-velocity-history.json`; compare the latest row against the previous row before posting to #leads._
|
||||
|
||||
## Post to #leads
|
||||
|
||||
```text
|
||||
FN-6612 weekly test velocity: gate 7.7s (+778ms), boot smoke 17.4s (-557ms), pnpm test 9.3s (-4.4s), quarantine ledger 1 (+1). Slowest file: packages/dashboard/src/__tests__/insights-routes.test.ts at 26.5s. Deletion-due quarantines: 0.
|
||||
FN-6612 weekly test velocity: gate 36.3s (+28.6s), boot smoke 25.3s (+7.9s), pnpm test 43.4s (+34.0s), quarantine ledger 1 (0). Slowest file: packages/dashboard/src/__tests__/insights-routes.test.ts at 26.5s. Deletion-due quarantines: 0.
|
||||
```
|
||||
|
||||
## How to refresh
|
||||
|
||||
@@ -1361,6 +1361,118 @@
|
||||
"measurementFailures": [],
|
||||
"timingSnapshotCapturedAt": "2026-06-27T05:41:42.568Z",
|
||||
"timingNotes": []
|
||||
},
|
||||
{
|
||||
"capturedAt": "2026-07-08T09:30:50.606Z",
|
||||
"gateMs": 36301,
|
||||
"bootSmokeMs": 25333,
|
||||
"testMs": 43360,
|
||||
"quarantineCount": 1,
|
||||
"slowestTop20": [
|
||||
{
|
||||
"file": "packages/dashboard/src/__tests__/insights-routes.test.ts",
|
||||
"ms": 26500,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/engine/src/runtimes/__tests__/in-process-runtime.test.ts",
|
||||
"ms": 24700,
|
||||
"package": "@fusion/engine"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/src/__tests__/workflow-routes.test.ts",
|
||||
"ms": 22000,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/core/src/__tests__/db.test.ts",
|
||||
"ms": 21200,
|
||||
"package": "@fusion/core"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/app/components/__tests__/GitManagerModal.test.tsx",
|
||||
"ms": 16900,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/core/src/__tests__/mission-store.test.ts",
|
||||
"ms": 16000,
|
||||
"package": "@fusion/core"
|
||||
},
|
||||
{
|
||||
"file": "packages/cli/src/__tests__/extension.test.ts",
|
||||
"ms": 15700,
|
||||
"package": "@runfusion/fusion"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/app/components/__tests__/AgentPromptsManager.test.tsx",
|
||||
"ms": 14800,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/app/components/__tests__/App.test.tsx",
|
||||
"ms": 14600,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/app/components/__tests__/TaskDetailModal.inline-editing-and-integrations.test.tsx",
|
||||
"ms": 14100,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/app/components/__tests__/TaskDetailModal.rendering.test.tsx",
|
||||
"ms": 13700,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/src/__tests__/routes-auth.test.ts",
|
||||
"ms": 13600,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/core/src/__tests__/agent-store.test.ts",
|
||||
"ms": 13400,
|
||||
"package": "@fusion/core"
|
||||
},
|
||||
{
|
||||
"file": "packages/engine/src/__tests__/workspace-merger-idempotency.test.ts",
|
||||
"ms": 12700,
|
||||
"package": "@fusion/engine"
|
||||
},
|
||||
{
|
||||
"file": "packages/engine/src/__tests__/self-healing-workspace.test.ts",
|
||||
"ms": 11800,
|
||||
"package": "@fusion/engine"
|
||||
},
|
||||
{
|
||||
"file": "packages/engine/src/__tests__/pr-response-run.test.ts",
|
||||
"ms": 11600,
|
||||
"package": "@fusion/engine"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/app/components/__tests__/ListView.test.tsx",
|
||||
"ms": 11300,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "plugins/fusion-plugin-compound-engineering/src/__tests__/sync.test.ts",
|
||||
"ms": 11000,
|
||||
"package": "@fusion-plugin-examples/compound-engineering"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/app/components/__tests__/AgentDetailView.settings.test.tsx",
|
||||
"ms": 10700,
|
||||
"package": "@fusion/dashboard"
|
||||
},
|
||||
{
|
||||
"file": "packages/dashboard/app/components/__tests__/SecretsView.test.tsx",
|
||||
"ms": 10700,
|
||||
"package": "@fusion/dashboard"
|
||||
}
|
||||
],
|
||||
"measurementFailures": [],
|
||||
"timingSnapshotCapturedAt": "2026-06-27T05:41:42.568Z",
|
||||
"timingNotes": []
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user