FN-6790: quiesce deferred task-created writes on close
Rescue task-document coverage by making TaskStore teardown stop late deferred task-created filesystem work. - Track deferred title write and task-created hook phases so close can drain active work. - Skip late deferred task-created work once a store is closing to avoid recreating removed fixture roots. - Update task-document regression coverage, harness teardown awaiting, and rescue documentation while keeping task-documents loaded. Files changed: docs/testing.md | 4 ++ packages/core/src/__tests__/store-test-helpers.ts | 2 +- packages/core/src/__tests__/task-documents.test.ts | 39 ++++++++++- packages/core/src/store.ts | 75 +++++++++++++++------- packages/core/vitest.config.ts | 3 + 5 files changed, 96 insertions(+), 27 deletions(-) Fusion-Task-Id: FN-6790 Fusion-Task-Lineage: c458fc4a-b2b4-4cc4-9d88-da8c43f6916d
This commit is contained in:
@@ -209,6 +209,10 @@ FNXC:CoreTests 2026-06-19-15:05: Merge verification re-observed store-concurrent
|
||||
|
||||
**2026-06-19 core suite-load rescue (FN-6741):** `activity-analytics.test.ts`, `db.test.ts`, `store-create-summarize-deferred-hook.test.ts`, `vitest-teardown-worker-root-cleanup.test.ts`, and `settings-export.test.ts` were rescued before their 2026-07-03 deletion deadline. The key evidence was a loaded `pnpm --filter @fusion/core test` pass across the package after re-quarantining `store-concurrent-writes.test.ts`, with no `ENOTEMPTY`, `EBUSY`, hook timeout, or missed deferred hook in the rescued files. Four files needed no weakening because their regression value still held under load; `settings-export.test.ts` kept its import/export coverage but now closes the real `TaskStore` before removing the fixture root. `store-concurrent-writes.test.ts` remains in the deletion ratchet after merge verification re-observed the broad-lane SQLite lock flake. Required closure evidence for this class is ledger/config convergence, the rescued package lane, the timeout-appeasement guard, `pnpm test:gate`, `pnpm test`, `pnpm typecheck`, and `pnpm build`.
|
||||
|
||||
<!-- FNXC:CoreTests 2026-06-20-05:28: FN-6790 proved a loaded @fusion/core ENOENT can come from TaskStore deferred task-created work that writes task.json after close while a fixture removes the root. Rescue this class by making close quiesce active deferred write/hook work and skip late work after closing; prove it with a controlled deferred-summarizer regression, loaded core lane, timeout-appeasement guard, and bounded temp-prefix output, not retries, timeouts, or worker reductions. -->
|
||||
|
||||
**2026-06-20 core task-documents rescue (FN-6790):** `packages/core/src/__tests__/task-documents.test.ts` stays loaded and unquarantined. The broad-lane symptom was an `ENOENT` during atomic `task.json` rename; the root-cause class is fire-and-forget `TaskStore` deferred task-created work (title summarization and task-created hook) entering an update after `store.close()` while the fixture root is being removed. The fix tracks active post-summarization write/hook work, makes `close()` mark the store as closing and await active work, and skips late deferred work that has not entered the write phase so intentionally stalled summarizers do not hang teardown. The regression test releases a controlled deferred summarizer only after close and root removal, then asserts the fixture root is not recreated. Closure evidence is targeted file coverage, a loaded `pnpm --filter @fusion/core test` pass, timeout-appeasement guard, bounded `fusion-test-workers-*`/`kb-task-docs-test-*` output, `pnpm test:gate`, `pnpm test`, `pnpm typecheck`, and `pnpm build`; no quarantine ledger/config entries or timeout/worker appeasement are allowed for this file.
|
||||
|
||||
<!-- FNXC:DashboardSessionTests 2026-06-19-16:19: FN-6742 proved dashboard session cross-tab coverage still catches real lock-holder regressions under mutation, but its route-only harness leaked TaskStore-backed `.fusion` cleanup work under a loaded shard. Rescue this class by disposing the API router, stopping scheduled session cleanup, closing stores/databases, and draining bounded check turns before removing the worker fixture; do not widen timeouts, add retries, or reduce worker load. -->
|
||||
|
||||
**2026-06-19 dashboard session-cross-tab rescue (FN-6742):** `packages/dashboard/src/__tests__/session-cross-tab.test.ts` was rescued before its 2026-07-03 deletion deadline. The loaded `dashboard-api-quality-backfill` shard reproduced the original `fusion-test-workers-*` `ENOTEMPTY` cleanup failure with the quarantine exclude temporarily removed, while the test's assertions retained value by failing when the expected lock holder was mutated from `tab-a` to `tab-z`. The fix keeps the test unquarantined by disposing the created API router, stopping `AiSessionStore` scheduled cleanup, closing the real `TaskStore`/SQLite handles, hiding route EventEmitter hooks not used by this harness, and draining four bounded check-phase turns before deleting the temp root. The ledger and `packages/dashboard/vitest.config.ts` exclude were updated in lockstep; later loaded runs no longer failed this file, and unrelated dashboard loaded-suite failures are tracked separately rather than weakening this test.
|
||||
|
||||
@@ -154,7 +154,7 @@ export function createTaskStoreTestHarness() {
|
||||
vi.useRealTimers();
|
||||
store.stopWatching();
|
||||
await delay(0);
|
||||
store.close();
|
||||
await store.close();
|
||||
await rm(rootDir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 });
|
||||
await rm(globalDir, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 });
|
||||
},
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from "vitest";
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
|
||||
import { existsSync, mkdtempSync } from "node:fs";
|
||||
import { rm } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
@@ -32,7 +32,7 @@ describe("TaskStore task documents", () => {
|
||||
|
||||
afterEach(async () => {
|
||||
try {
|
||||
store.close();
|
||||
await store.close();
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
@@ -62,6 +62,39 @@ describe("TaskStore task documents", () => {
|
||||
expect(index?.name).toBe("idxTaskDocumentsTaskKey");
|
||||
});
|
||||
|
||||
it("does not let deferred title work recreate a removed fixture root after close", async () => {
|
||||
/*
|
||||
FNXC:CoreTests 2026-06-20-05:18:
|
||||
FN-6790 rescued task-documents under the loaded core lane by proving TaskStore.close() closes the race between deferred task-created background work and per-test root removal. The test must keep soft-delete document assertions intact while guarding the shared teardown invariant that late title summarization cannot write task.json after close.
|
||||
*/
|
||||
let releaseSummarize!: (title: string) => void;
|
||||
const summarizeStarted = vi.fn();
|
||||
const summarizeTitle = new Promise<string>((resolve) => {
|
||||
releaseSummarize = resolve;
|
||||
});
|
||||
|
||||
const task = await store.createTask(
|
||||
{ description: "a".repeat(201) },
|
||||
{
|
||||
onSummarize: vi.fn(async () => {
|
||||
summarizeStarted();
|
||||
return summarizeTitle;
|
||||
}),
|
||||
settings: { autoSummarizeTitles: true },
|
||||
},
|
||||
);
|
||||
|
||||
await vi.waitFor(() => expect(summarizeStarted).toHaveBeenCalled());
|
||||
await store.close();
|
||||
await rm(rootDir, { recursive: true, force: true });
|
||||
|
||||
releaseSummarize("Late title after close");
|
||||
await sleep(10);
|
||||
|
||||
expect(existsSync(rootDir)).toBe(false);
|
||||
expect(task.title).toBeUndefined();
|
||||
});
|
||||
|
||||
it("creates a document with revision 1, default author, and optional metadata", async () => {
|
||||
const task = await store.createTask({ description: "Document task" });
|
||||
|
||||
|
||||
@@ -1522,6 +1522,23 @@ export class TaskStore extends EventEmitter<TaskStoreEvents> {
|
||||
private debounceMs = 150;
|
||||
/** Per-task promise chain for serializing writes */
|
||||
private taskLocks: Map<string, Promise<void>> = new Map();
|
||||
private closing = false;
|
||||
private deferredTaskCreatedWork = new Set<Promise<void>>();
|
||||
/**
|
||||
* FNXC:CoreTests 2026-06-20-05:17:
|
||||
* Core loaded-suite teardown may remove a per-test project root while createTask's deferred title summarization or task-created hook is still writing task.json. Track only the post-summarization write/hook phase so close() can quiesce active filesystem mutations without hanging on intentionally stalled summarizer prompts.
|
||||
*/
|
||||
private trackDeferredTaskCreatedWork(work: () => Promise<void>): Promise<void> {
|
||||
if (this.closing) return Promise.resolve();
|
||||
const promise = (async () => {
|
||||
if (this.closing) return;
|
||||
await work();
|
||||
})();
|
||||
this.deferredTaskCreatedWork.add(promise);
|
||||
return promise.finally(() => {
|
||||
this.deferredTaskCreatedWork.delete(promise);
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Cross-task lock for worktree path allocation. Serializes the
|
||||
* read-tasks → pick-name → write-task sequence so two concurrent
|
||||
@@ -1774,6 +1791,7 @@ export class TaskStore extends EventEmitter<TaskStoreEvents> {
|
||||
}
|
||||
|
||||
async init(): Promise<void> {
|
||||
this.closing = false;
|
||||
await mkdir(this.tasksDir, { recursive: true });
|
||||
|
||||
// U4: register the default-workflow trait hook implementations into the
|
||||
@@ -4399,14 +4417,17 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS}
|
||||
const generatedTitle = await onSummarize!(input.description);
|
||||
const sanitizedTitle = sanitizeTitle(generatedTitle);
|
||||
if (sanitizedTitle) {
|
||||
const currentTask = this.readTaskFromDb(id);
|
||||
if (currentTask && !currentTask.title) {
|
||||
// FN-5077: normalizeTitleForTaskId may return null for dangling fragments; only persist usable titles.
|
||||
const normalizedTitle = normalizeTitleForTaskId(sanitizedTitle, id);
|
||||
if (normalizedTitle.title) {
|
||||
await this.updateTask(id, { title: normalizedTitle.title });
|
||||
await this.trackDeferredTaskCreatedWork(async () => {
|
||||
if (this.closing) return;
|
||||
const currentTask = this.readTaskFromDb(id);
|
||||
if (currentTask && !currentTask.title) {
|
||||
// FN-5077: normalizeTitleForTaskId may return null for dangling fragments; only persist usable titles.
|
||||
const normalizedTitle = normalizeTitleForTaskId(sanitizedTitle, id);
|
||||
if (normalizedTitle.title && !this.closing) {
|
||||
await this.updateTask(id, { title: normalizedTitle.title });
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
const autoEnabled = resolvedSettings?.autoSummarizeTitles === true;
|
||||
@@ -4422,22 +4443,26 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS}
|
||||
);
|
||||
}
|
||||
|
||||
let latestTask = task;
|
||||
try {
|
||||
const refreshed = this.readTaskFromDb(id);
|
||||
if (refreshed) latestTask = refreshed;
|
||||
} catch {
|
||||
// Best-effort refresh; fall back to original task snapshot.
|
||||
}
|
||||
await this.trackDeferredTaskCreatedWork(async () => {
|
||||
if (this.closing) return;
|
||||
let latestTask = task;
|
||||
try {
|
||||
const refreshed = this.readTaskFromDb(id);
|
||||
if (refreshed) latestTask = refreshed;
|
||||
} catch {
|
||||
// Best-effort refresh; fall back to original task snapshot.
|
||||
}
|
||||
|
||||
try {
|
||||
await this.invokeTaskCreatedHook(latestTask);
|
||||
} catch (err) {
|
||||
storeLog.warn("Deferred task-created hook failed", {
|
||||
taskId: id,
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
});
|
||||
}
|
||||
if (this.closing) return;
|
||||
try {
|
||||
await this.invokeTaskCreatedHook(latestTask);
|
||||
} catch (err) {
|
||||
storeLog.warn("Deferred task-created hook failed", {
|
||||
taskId: id,
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
});
|
||||
}
|
||||
});
|
||||
}).catch((err) => {
|
||||
const autoEnabled = resolvedSettings?.autoSummarizeTitles === true;
|
||||
storeLog.error("Unexpected title summarization promise-chain failure", {
|
||||
@@ -15671,7 +15696,11 @@ ${stepsSection}`;
|
||||
* Close the database connection and clean up resources.
|
||||
* Call this when the store is no longer needed (e.g., short-lived per-request stores).
|
||||
*/
|
||||
close(): void {
|
||||
async close(): Promise<void> {
|
||||
this.closing = true;
|
||||
if (this.deferredTaskCreatedWork.size > 0) {
|
||||
await Promise.allSettled([...this.deferredTaskCreatedWork]);
|
||||
}
|
||||
this.stopWatching();
|
||||
// Flush any remaining buffered agent log entries before closing.
|
||||
// Wrap in try-catch because entries for already-deleted tasks will fail FK check.
|
||||
|
||||
@@ -39,6 +39,9 @@ const quarantinedCoreTests = [
|
||||
|
||||
FNXC:CoreTests 2026-06-19-15:05:
|
||||
Merge verification for FN-6741 observed store-concurrent-writes fail again under the broad @fusion/core lane with SQLite BEGIN IMMEDIATE lock exhaustion. Re-quarantine that single suite-load lock flake in lockstep with the ledger; keep the other rescued core files loaded.
|
||||
|
||||
FNXC:CoreTests 2026-06-20-05:19:
|
||||
FN-6790 found no task-documents quarantine half-state on HEAD and rescued the ENOENT-rename class by quiescing deferred task-created write/hook work on TaskStore.close(). Keep task-documents loaded; do not add a ledger/config exclude unless a new loaded run fails after this lifecycle seam is ruled out.
|
||||
*/
|
||||
"src/__tests__/store-concurrent-writes.test.ts",
|
||||
];
|
||||
|
||||
Reference in New Issue
Block a user