From c18e827b90b97e8fc8653e55072f084b0f908484 Mon Sep 17 00:00:00 2001 From: gsxdsm Date: Sun, 21 Jun 2026 06:22:23 -0700 Subject: [PATCH] FN-6839: await CLI cached store shutdown Rescue the retained CLI quarantine by making TaskStore shutdown deterministic before fixture cleanup. - Await cached extension TaskStore closes and clear the cache before asynchronous shutdown drains. - Update CLI fixtures to await direct and cached store closure, with a regression proving cached close is awaited. - Remove rescued CLI quarantine entries/excludes and document the loaded-suite rescue with a patch changeset. Files changed: .changeset/fn-6839-close-cached-stores.md | 5 ++ CLAUDE.md | 1 + docs/testing.md | 4 ++ .../extension-agent-set-instructions.test.ts | 2 +- .../__tests__/extension-goal-tools-audit.test.ts | 2 +- .../cli/src/__tests__/extension-goal-tools.test.ts | 2 +- .../cli/src/__tests__/extension-insights.test.ts | 6 +-- .../__tests__/extension-mission-goal-tools.test.ts | 2 +- .../cli/src/__tests__/extension-task-tools.test.ts | 16 ++++--- packages/cli/src/__tests__/extension.test.ts | 56 +++++++++++----------- .../src/__tests__/research-extension-tools.test.ts | 4 +- packages/cli/src/extension.ts | 16 +++---- packages/cli/vitest.config.ts | 6 +-- scripts/lib/test-quarantine.json | 15 ------ 14 files changed, 68 insertions(+), 69 deletions(-) Fusion-Task-Id: FN-6839 Fusion-Task-Lineage: ea0c7c77-bb59-4db4-931d-5e052a1043e5 --- .changeset/fn-6839-close-cached-stores.md | 5 ++ CLAUDE.md | 1 + docs/testing.md | 4 ++ .../extension-agent-set-instructions.test.ts | 2 +- .../extension-goal-tools-audit.test.ts | 2 +- .../__tests__/extension-goal-tools.test.ts | 2 +- .../src/__tests__/extension-insights.test.ts | 6 +- .../extension-mission-goal-tools.test.ts | 2 +- .../__tests__/extension-task-tools.test.ts | 16 ++++-- packages/cli/src/__tests__/extension.test.ts | 56 +++++++++---------- .../research-extension-tools.test.ts | 4 +- packages/cli/src/extension.ts | 16 +++--- packages/cli/vitest.config.ts | 6 +- scripts/lib/test-quarantine.json | 15 ----- 14 files changed, 68 insertions(+), 69 deletions(-) create mode 100644 .changeset/fn-6839-close-cached-stores.md create mode 100644 CLAUDE.md diff --git a/.changeset/fn-6839-close-cached-stores.md b/.changeset/fn-6839-close-cached-stores.md new file mode 100644 index 0000000000..ac3b5577de --- /dev/null +++ b/.changeset/fn-6839-close-cached-stores.md @@ -0,0 +1,5 @@ +--- +"@runfusion/fusion": patch +--- + +Await CLI extension cached TaskStore shutdown so deferred filesystem writes and SQLite handles drain before fixture or process cleanup. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000000..43c994c2d3 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/docs/testing.md b/docs/testing.md index 9dc46c3a09..8d277f604d 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -217,6 +217,10 @@ FNXC:CoreTests 2026-06-19-15:05: Merge verification re-observed store-concurrent **2026-06-20 residual CLI quarantine triage (FN-6795):** the six `2026-06-19` residual entries were temporarily unexcluded and exercised under targeted and loaded lanes. `store-concurrent-writes.test.ts`, `extension-goal-tools.test.ts`, `extension-mission-goal-tools.test.ts`, and `research-extension-tools.test.ts` were rescued because their direct and package/gate lanes stayed green with no `ENOTEMPTY`, lock exhaustion, or cross-test state drift. The final full `@runfusion/fusion` lane still timed out `extension-task-tools.test.ts`, `extension.test.ts`, and a newly observed `bin.test.ts` case only under package load while the focused rerun passed, so those files remain quarantined in ledger/config lockstep with the original 2026-07-03 deletion deadline for the two 2026-06-19 residuals. Treat future work as a fixture-load root-cause search, not timeout/retry/worker appeasement. + + +**2026-06-21 retained CLI quarantine rescue (FN-6839):** `bin.test.ts`, `extension-task-tools.test.ts`, and `extension.test.ts` were rescued before their 2026-07-03/2026-07-04 deletion deadlines. The failed prior attempt to skip `task:created` hooks ruled out a hook-only root cause; the real reusable invariant is that `TaskStore.close()` is async and must be awaited for both extension cached stores and direct fixture stores before removing temp roots, otherwise deferred filesystem work and SQLite/WAL handles can survive under loaded `@runfusion/fusion` workers. `closeCachedStores()` now awaits each cached store close, the quarantined fixtures await direct/cached shutdown, and the extension regression test asserts cached shutdown does not resolve before async close settles. The three ledger entries and `packages/cli/vitest.config.ts` excludes were removed in lockstep; the grouped three-file lane and full CLI package lane pass unexcluded with no hook/body timeout, no `ENOTEMPTY`/`EBUSY`, and no timeout/retry/worker appeasement. The broader `pnpm test` command is currently blocked before tests by unrelated line-count guardrail failures tracked by FN-6849, not by these rescued CLI files. + **2026-06-19 dashboard session-cross-tab rescue (FN-6742):** `packages/dashboard/src/__tests__/session-cross-tab.test.ts` was rescued before its 2026-07-03 deletion deadline. The loaded `dashboard-api-quality-backfill` shard reproduced the original `fusion-test-workers-*` `ENOTEMPTY` cleanup failure with the quarantine exclude temporarily removed, while the test's assertions retained value by failing when the expected lock holder was mutated from `tab-a` to `tab-z`. The fix keeps the test unquarantined by disposing the created API router, stopping `AiSessionStore` scheduled cleanup, closing the real `TaskStore`/SQLite handles, hiding route EventEmitter hooks not used by this harness, and draining four bounded check-phase turns before deleting the temp root. The ledger and `packages/dashboard/vitest.config.ts` exclude were updated in lockstep; later loaded runs no longer failed this file, and unrelated dashboard loaded-suite failures are tracked separately rather than weakening this test. diff --git a/packages/cli/src/__tests__/extension-agent-set-instructions.test.ts b/packages/cli/src/__tests__/extension-agent-set-instructions.test.ts index 7efbf78de1..4db294f3de 100644 --- a/packages/cli/src/__tests__/extension-agent-set-instructions.test.ts +++ b/packages/cli/src/__tests__/extension-agent-set-instructions.test.ts @@ -58,7 +58,7 @@ async function withOrg( ids: { manager: manager.id, middle: middle.id, leaf: leaf.id, peer: peer.id }, }); } finally { - closeCachedStores(); + await closeCachedStores(); agentStore.close(); await rm(cwd, { recursive: true, force: true }); } diff --git a/packages/cli/src/__tests__/extension-goal-tools-audit.test.ts b/packages/cli/src/__tests__/extension-goal-tools-audit.test.ts index 6d85062a08..7a63c51ec4 100644 --- a/packages/cli/src/__tests__/extension-goal-tools-audit.test.ts +++ b/packages/cli/src/__tests__/extension-goal-tools-audit.test.ts @@ -32,7 +32,7 @@ describe("extension goal tools retrieval audit", () => { }); afterEach(async () => { - closeCachedStores(); + await closeCachedStores(); await rm(tmpDir, { recursive: true, force: true }); vi.restoreAllMocks(); }); diff --git a/packages/cli/src/__tests__/extension-goal-tools.test.ts b/packages/cli/src/__tests__/extension-goal-tools.test.ts index af5a216f2f..6eb317d5d8 100644 --- a/packages/cli/src/__tests__/extension-goal-tools.test.ts +++ b/packages/cli/src/__tests__/extension-goal-tools.test.ts @@ -59,7 +59,7 @@ describe("extension goal retrieval tools", () => { }); afterEach(async () => { - closeCachedStores(); + await closeCachedStores(); await rm(tmpDir, { recursive: true, force: true }); }); diff --git a/packages/cli/src/__tests__/extension-insights.test.ts b/packages/cli/src/__tests__/extension-insights.test.ts index 67585c5b72..afa5949cd8 100644 --- a/packages/cli/src/__tests__/extension-insights.test.ts +++ b/packages/cli/src/__tests__/extension-insights.test.ts @@ -45,7 +45,7 @@ describe("fn insight extension tools", () => { }); afterEach(async () => { - closeCachedStores(); + await closeCachedStores(); await rm(tmpDir, { recursive: true, force: true }); }); @@ -68,7 +68,7 @@ describe("fn insight extension tools", () => { provenance: { trigger: "manual" }, content: "Ensure this appears in extension output", }); - store.close(); + await store.close(); const listTool = api.tools.get("fn_insight_list")!; const listResult = await listTool.execute("call-1", { category: "quality" }, undefined, undefined, makeCtx(tmpDir)); @@ -88,7 +88,7 @@ describe("fn insight extension tools", () => { const run = insightStore.createRun("", { trigger: "manual" }); insightStore.updateRun(run.id, { status: "completed", insightsCreated: 2, insightsUpdated: 1 }); - store.close(); + await store.close(); const listTool = api.tools.get("fn_insight_run_list")!; const listResult = await listTool.execute("call-3", { status: "completed" }, undefined, undefined, makeCtx(tmpDir)); diff --git a/packages/cli/src/__tests__/extension-mission-goal-tools.test.ts b/packages/cli/src/__tests__/extension-mission-goal-tools.test.ts index c89d95da67..093332286a 100644 --- a/packages/cli/src/__tests__/extension-mission-goal-tools.test.ts +++ b/packages/cli/src/__tests__/extension-mission-goal-tools.test.ts @@ -54,7 +54,7 @@ describe("extension mission goal tools", () => { }); afterEach(async () => { - closeCachedStores(); + await closeCachedStores(); await rm(tmpDir, { recursive: true, force: true }); }); diff --git a/packages/cli/src/__tests__/extension-task-tools.test.ts b/packages/cli/src/__tests__/extension-task-tools.test.ts index d517332342..f85c06cdba 100644 --- a/packages/cli/src/__tests__/extension-task-tools.test.ts +++ b/packages/cli/src/__tests__/extension-task-tools.test.ts @@ -20,7 +20,7 @@ function makeCtx(cwd: string) { return { cwd } as any; } -let closeLoadedExtensionStores: (() => void) | undefined; +let closeLoadedExtensionStores: (() => Promise) | undefined; async function loadExtension() { const mod = await import("../extension.js"); @@ -37,8 +37,12 @@ describe("extension task tools resolve repo root from worktrees", () => { vi.resetModules(); }); - afterEach(() => { - closeLoadedExtensionStores?.(); + afterEach(async () => { + /* + FNXC:CliTests 2026-06-21-09:58: + FN-6839 requires canonical-root fixture cleanup to await cached and direct TaskStore shutdown before temp roots are removed; this preserves the loaded-lane rescue without timeout or worker appeasement. + */ + await closeLoadedExtensionStores?.(); closeLoadedExtensionStores = undefined; vi.restoreAllMocks(); vi.doUnmock("@fusion/core"); @@ -86,7 +90,7 @@ describe("extension task tools resolve repo root from worktrees", () => { expect(show.content[0].text).toContain("Task from canonical root"); expect(list.content[0].text).toContain(created.id); } finally { - store?.close(); + await store?.close(); await rm(repoRoot, { recursive: true, force: true }); } }); @@ -130,7 +134,7 @@ describe("extension task tools resolve repo root from worktrees", () => { expect(show.content[0].text).toContain("Task visible from merge worktree"); expect(list.content[0].text).toContain(created.id); } finally { - store?.close(); + await store?.close(); try { git(repoRoot, `worktree remove --force ${JSON.stringify(mergeRoot)}`); } catch { @@ -185,7 +189,7 @@ describe("extension task tools resolve repo root from worktrees", () => { expect(show.content[0]?.text).toContain(created.id); expect(warnSpy).toHaveBeenCalledTimes(1); } finally { - store?.close(); + await store?.close(); await rm(repoRoot, { recursive: true, force: true }); } }); diff --git a/packages/cli/src/__tests__/extension.test.ts b/packages/cli/src/__tests__/extension.test.ts index e3273fc61e..1d28b7e073 100644 --- a/packages/cli/src/__tests__/extension.test.ts +++ b/packages/cli/src/__tests__/extension.test.ts @@ -123,7 +123,7 @@ async function seedWorkflow(cwd: string, name = "QA workflow"): Promise const workflow = await store.createWorkflowDefinition({ name, ir: linearWorkflowIr(name) }); return workflow.id; } finally { - store.close(); + await store.close(); } } @@ -135,7 +135,7 @@ async function readTaskWorkflowState(cwd: string, taskId: string) { const selection = store.getTaskWorkflowSelection(taskId); return { task, selection }; } finally { - store.close(); + await store.close(); } } @@ -246,7 +246,7 @@ describe.skipIf(!SHOULD_RUN_LEGACY_EXTENSION_INTEGRATION)("fn pi extension (lega }); afterEach(async () => { - closeCachedStores(); + await closeCachedStores(); await removeDirWithRetries(tmpDir); }); @@ -406,7 +406,7 @@ describe.skipIf(!SHOULD_RUN_LEGACY_EXTENSION_INTEGRATION)("fn pi extension (lega try { await store.setDefaultWorkflowId(workflowId); } finally { - store.close(); + await store.close(); } const tool = api.tools.get("fn_task_create")!; @@ -2439,7 +2439,7 @@ describe.skipIf(!SHOULD_RUN_LEGACY_EXTENSION_INTEGRATION)("fn pi extension (lega url: "https://github.com/acme/demo/issues/1", }, }); - store.close(); + await store.close(); const tool = api.tools.get("fn_task_import_github")!; vi.mocked(runGhJsonAsync).mockResolvedValueOnce([ @@ -2471,7 +2471,7 @@ describe.skipIf(!SHOULD_RUN_LEGACY_EXTENSION_INTEGRATION)("fn pi extension (lega url: "https://github.com/acme/demo/issues/1", }, }); - store.close(); + await store.close(); const tool = api.tools.get("fn_task_import_github_issue")!; vi.mocked(runGhJsonAsync).mockResolvedValueOnce({ @@ -2554,27 +2554,27 @@ describe("fn pi extension (runnable structured-output regression slice)", () => */ for (const store of openStores.splice(0)) { try { - store.close(); + await store.close(); } catch { // Best effort: close all real stores before removing fixture roots. } } - closeCachedStores(); + await closeCachedStores(); await removeDirWithRetries(tmpDir); }); - it("closes cached TaskStore handles before fixture removal (FN-6734 regression)", async () => { - /* - FNXC:CliTests 2026-06-19-11:35: - FN-6734 needs a deterministic guard for the close-before-remove invariant: extension tools cache a real TaskStore, so cleanup must close cached stores before removing the fixture root. - */ - const closeSpy = vi.spyOn(TaskStore.prototype, "close"); - const createTool = api.tools.get("fn_task_create")!; - - await createTool.execute("close-before-remove", { description: "seed cached store" }, undefined, undefined, makeCtx(tmpDir)); - closeCachedStores(); - + it("closes cached TaskStore handles before fixture removal (FN-6734/FN-6839 regression)", async () => { + /* FNXC:CliTests 2026-06-21-09:58: FN-6839 extends FN-6734 from "close was called" to "close was awaited" because TaskStore.close() quiesces deferred task-created writes before SQLite/WAL handles are safe to remove under loaded CLI lanes. */ + const originalClose = TaskStore.prototype.close; let closeSettled = false; + const closeSpy = vi.spyOn(TaskStore.prototype, "close").mockImplementation(async function (this: TaskStore) { + await new Promise((resolve) => setImmediate(resolve)); + await originalClose.call(this); + closeSettled = true; + }); + await api.tools.get("fn_task_create")!.execute("close-before-remove", { description: "seed cached store" }, undefined, undefined, makeCtx(tmpDir)); + await closeCachedStores(); expect(closeSpy).toHaveBeenCalled(); + expect(closeSettled).toBe(true); await expect(rm(tmpDir, { recursive: true, force: true })).resolves.not.toThrow(); tmpDir = await mkdtemp(join(tmpdir(), "kb-ext-fast-")); }); @@ -2618,7 +2618,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => await store.createTask({ description: "Planning task one" }); await store.createTask({ description: "Todo task one", column: "todo" }); } finally { - store.close(); + await store.close(); } const listTool = api.tools.get("fn_task_list")!; @@ -2639,7 +2639,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => try { await store.createTask({ description: "Finished task keeps the board non-empty", column: "done" }); } finally { - store.close(); + await store.close(); } const listTool = api.tools.get("fn_task_list")!; @@ -2671,7 +2671,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => const first = await store.createTask({ description: "Small todo task one", column: "todo" }); await store.createTask({ description: "Small todo task two", column: "todo", dependencies: [first.id] }); } finally { - store.close(); + await store.close(); } const listTool = api.tools.get("fn_task_list")!; @@ -2729,7 +2729,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => }); } } finally { - store.close(); + await store.close(); } const listTool = api.tools.get("fn_task_list")!; @@ -2790,7 +2790,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => }); } } finally { - store.close(); + await store.close(); } const listTool = api.tools.get("fn_task_list")!; @@ -2832,7 +2832,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => }); } } finally { - store.close(); + await store.close(); } const listTool = api.tools.get("fn_task_list")!; @@ -2893,7 +2893,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => }); } } finally { - store.close(); + await store.close(); } vi.resetModules(); @@ -3861,7 +3861,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => expect(result.content[0].text).toContain("Start the project engine to process pending runs"); expect(result.details.status).toBe("queued"); } finally { - store.close(); + await store.close(); } }); @@ -3911,7 +3911,7 @@ describe("fn pi extension (runnable structured-output regression slice)", () => expect(result.details.summary).toBe("done"); expect(result.content[0].text).toContain("is completed"); } finally { - store.close(); + await store.close(); } }); }); diff --git a/packages/cli/src/__tests__/research-extension-tools.test.ts b/packages/cli/src/__tests__/research-extension-tools.test.ts index a23db3a629..9134078153 100644 --- a/packages/cli/src/__tests__/research-extension-tools.test.ts +++ b/packages/cli/src/__tests__/research-extension-tools.test.ts @@ -59,12 +59,12 @@ describe("research extension tools", () => { */ for (const store of openStores.splice(0)) { try { - store.close(); + await store.close(); } catch { // Best effort: cleanup must continue so the temp root can be removed. } } - closeCachedStores(); + await closeCachedStores(); await rm(tmpDir, { recursive: true, force: true }); }); diff --git a/packages/cli/src/extension.ts b/packages/cli/src/extension.ts index c50b590c9d..3ee502c512 100644 --- a/packages/cli/src/extension.ts +++ b/packages/cli/src/extension.ts @@ -158,20 +158,20 @@ async function getStore(cwd: string): Promise { } /** @internal Exposed so tests and the extension shutdown hook can close cached stores deterministically; not a public CLI API contract. */ -export function closeCachedStores(): void { +export async function closeCachedStores(): Promise { /* - FNXC:CliTests 2026-06-17-23:58: - FN-6626 found the CLI extension cache cleared real TaskStore instances without closing them, leaving SQLite/WAL handles to survive module resets and making canonical-project-root task-tool tests timeout under suite load. - Close every cached store deterministically on extension shutdown and in tests; do not appease the load-sensitive seam with timeouts, retries, or worker changes. + FNXC:CliTests 2026-06-17-23:58: FN-6626 found the CLI extension cache cleared real TaskStore instances without closing them, leaving SQLite/WAL handles to survive module resets and making canonical-project-root task-tool tests timeout under suite load. + FNXC:CliTests 2026-06-21-09:58: FN-6839 requires awaiting TaskStore.close() so deferred task-created filesystem work and SQLite/WAL handles drain before temp-root removal; do not appease this loaded-lane seam with timeouts, retries, or worker changes. */ - for (const store of storeCache.values()) { + const stores = [...storeCache.values()]; + storeCache.clear(); + for (const store of stores) { try { - store.close(); + await store.close(); } catch (error) { console.warn("[fusion-extension] cached TaskStore close skipped", error); } } - storeCache.clear(); } function getFusionDir(cwd: string): string { @@ -4699,6 +4699,6 @@ export default function kbExtension(pi: ExtensionAPI) { dashboardProcess = null; dashboardPort = null; } - closeCachedStores(); + void closeCachedStores(); }); } diff --git a/packages/cli/vitest.config.ts b/packages/cli/vitest.config.ts index d056c77e6e..83d8a984e5 100644 --- a/packages/cli/vitest.config.ts +++ b/packages/cli/vitest.config.ts @@ -33,10 +33,10 @@ const quarantinedCliTests: string[] = [ FNXC:CliTests 2026-06-20-10:04: FN-6795 final loaded verification re-exposed extension-task-tools, extension.test's built-dist-barrel case, and bin's no-args dashboard launch as package-lane-only timeouts while targeted reruns passed. Retain/quarantine these files in lockstep with the ledger rather than widening 5s/15s timeouts, adding retries, or changing worker budgets; the 2026-06-19 entries still delete on 2026-07-03 unless a real fixture-load invariant is found. + + FNXC:CliTests 2026-06-21-09:58: + FN-6839 rescues the retained bin, extension-task-tools, and extension suites by awaiting async TaskStore/cache shutdown before temp-root cleanup and proving the grouped/package lanes can run unexcluded. Keep the exclude list empty in lockstep with scripts/lib/test-quarantine.json; do not re-quarantine this loaded-lane signature without a new root-cause invariant. */ - "src/__tests__/bin.test.ts", - "src/__tests__/extension-task-tools.test.ts", - "src/__tests__/extension.test.ts", ]; export default defineConfig({ diff --git a/scripts/lib/test-quarantine.json b/scripts/lib/test-quarantine.json index 0fe4bf00d8..fa7167159a 100644 --- a/scripts/lib/test-quarantine.json +++ b/scripts/lib/test-quarantine.json @@ -5,21 +5,6 @@ "file": "packages/dashboard/src/__tests__/session-cross-tab.test.ts", "reason": "FN-6690 local workspace `pnpm test` observed ENOTEMPTY while removing the test's temp .fusion directory in dashboard-api-quality-backfill shard; isolated rerun passed, indicating cleanup flake rather than a lazy-view CSS regression.", "quarantinedAt": "2026-06-19" - }, - { - "file": "packages/cli/src/__tests__/bin.test.ts", - "reason": "FN-6795 final @runfusion/fusion package verification observed `launches dashboard when no args are provided` time out at the existing 15s budget only under the full package lane; a targeted rerun with the failing CLI files passed, so quarantine this newly observed suite-load flake on sight instead of widening timeouts or worker budgets.", - "quarantinedAt": "2026-06-20" - }, - { - "file": "packages/cli/src/__tests__/extension-task-tools.test.ts", - "reason": "FN-6795 loaded @runfusion/fusion package verification after attempted rescue still timed out both canonical-root task-tool cases only under the full package lane while the targeted rerun with bin/extension-task/extension passed; retain under the original 2026-06-19 deletion ratchet instead of widening the 5s test timeout, adding retries, or changing workers.", - "quarantinedAt": "2026-06-19" - }, - { - "file": "packages/cli/src/__tests__/extension.test.ts", - "reason": "FN-6795 loaded @runfusion/fusion package verification after attempted rescue still timed out the built-dist-barrel fn_task_list case only under the full package lane while the targeted rerun with bin/extension-task/extension passed; retain under the original 2026-06-19 deletion ratchet instead of widening the 5s test timeout or weakening assertions.", - "quarantinedAt": "2026-06-19" } ] }