FN-6421: quarantine flaky CLI integration tests
Quarantine load-sensitive CLI vitest suites instead of widening default test timeouts. - Add extension agent provisioning and serve suites to the CLI vitest quarantine list. - Record quarantine ledger entries with standalone pass evidence and deletion-clock dates. - Keep FNXC rationale next to the excluded CLI tests for future cleanup. Files changed: packages/cli/vitest.config.ts | 6 ++++++ scripts/lib/test-quarantine.json | 10 ++++++++++ 2 files changed, 16 insertions(+) Fusion-Task-Id: FN-6421 Fusion-Task-Lineage: de6db6a5-aad7-4e25-9bf4-7734dc6d8d05
This commit is contained in:
@@ -21,9 +21,14 @@ const quarantinedCliTests = [
|
||||
FNXC:CliTests 2026-06-14-01:58:
|
||||
mission.test includes a real temp-project end-to-end mission-goal case that exceeds the default 5s CLI timeout even as a standalone targeted run, then passes only when given 30s.
|
||||
Quarantine the slow file rather than encoding a longer timeout into the default package lane.
|
||||
|
||||
FNXC:CliTests 2026-06-13-20:05:
|
||||
FN-6421 quarantines the remaining FN-6419 CLI lane offenders after standalone evidence showed the agent-provisioning and serve suites pass directly but are integration-heavy under package-wide load.
|
||||
Keep them on the 14-day deletion clock rather than widening CLI test timeouts or loosening assertions.
|
||||
*/
|
||||
"src/__tests__/bin.test.ts",
|
||||
"src/__tests__/extension.test.ts",
|
||||
"src/__tests__/extension-agent-provisioning.test.ts",
|
||||
"src/__tests__/extension-experiment-finalize.test.ts",
|
||||
"src/__tests__/extension-github-tracking.test.ts",
|
||||
"src/__tests__/extension-goal-tools.test.ts",
|
||||
@@ -44,6 +49,7 @@ const quarantinedCliTests = [
|
||||
"src/commands/__tests__/ensure-project-registered.test.ts",
|
||||
"src/commands/__tests__/init.test.ts",
|
||||
"src/commands/__tests__/plugin.test.ts",
|
||||
"src/commands/__tests__/serve.test.ts",
|
||||
];
|
||||
|
||||
export default defineConfig({
|
||||
|
||||
@@ -170,6 +170,16 @@
|
||||
"file": "packages/cli/src/commands/__tests__/plugin.test.ts",
|
||||
"reason": "Slow/flaky CLI lane observed during `pnpm test` on 2026-06-14: full `@runfusion/fusion` package run timed out in `writes runPluginInstall metadata to central tables only` and leaked cross-test plugin path state into `includes getRootDir on the plugin loader taskStore mock`. A smaller direct run with agent-import/dashboard/plugin passed immediately afterward (`pnpm --filter @runfusion/fusion exec vitest run src/commands/__tests__/agent-import.test.ts src/commands/__tests__/dashboard.test.ts src/commands/__tests__/plugin.test.ts --silent=passed-only --reporter=dot`, 112/112), indicating suite-load/order sensitivity rather than a confirmed product bug.",
|
||||
"quarantinedAt": "2026-06-14"
|
||||
},
|
||||
{
|
||||
"file": "packages/cli/src/__tests__/extension-agent-provisioning.test.ts",
|
||||
"reason": "Slow/flaky CLI lane offender observed during FN-6419 broad `pnpm test` / targeted @runfusion/fusion verification: the extension agent provisioning suite exercises real temp projects and privileged `fn_agent_create`/`fn_agent_delete` extension tools, making it sensitive to package-wide CLI load and temp cleanup races. FN-6421 local cross-check after install found the current quarantined CLI lane green, and the two-offender direct run passed immediately (`pnpm --filter @runfusion/fusion exec vitest run src/__tests__/extension-agent-provisioning.test.ts src/commands/__tests__/serve.test.ts --silent=passed-only --reporter=dot`, 55/55), so this is quarantined from the default lane rather than appeased with broader timeouts.",
|
||||
"quarantinedAt": "2026-06-13"
|
||||
},
|
||||
{
|
||||
"file": "packages/cli/src/commands/__tests__/serve.test.ts",
|
||||
"reason": "Slow/flaky CLI lane offender observed during FN-6419 broad `pnpm test` / targeted @runfusion/fusion verification: the serve command suite is a large multi-project integration harness with mocked constructible engine classes, EventEmitter routing, timers, and temp directories, making it sensitive to package-wide CLI load and cleanup races. FN-6421 local cross-check after install found the current quarantined CLI lane green, and the two-offender direct run passed immediately (`pnpm --filter @runfusion/fusion exec vitest run src/__tests__/extension-agent-provisioning.test.ts src/commands/__tests__/serve.test.ts --silent=passed-only --reporter=dot`, 55/55), so this is quarantined from the default lane rather than appeased with wider test timeouts or loosened assertions.",
|
||||
"quarantinedAt": "2026-06-13"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user