root/fusion
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fix: make per-task API resilient to unreadable PROMPT.md; evict archived badge snapshots

Browse Source 
#5 root cause (reproduced): getTask — the shared load for the entire per-task
API — plus the mutation helpers updateTaskUnlocked, updateStep,
readPromptForArchive, and resetPromptCheckboxes all read PROMPT.md unguarded.
An unreadable PROMPT.md (root-owned from a prior `sudo` run -> EACCES, PROMPT.md
being a directory -> EISDIR, transient FS error) threw and 500'd every per-task
operation (GET/DELETE/PATCH/retry/reset/archive) for every task, while the
PROMPT.md-free board list and create kept working. These reads are now
best-effort: degrade (empty prompt / unsynced steps / skipped cosmetic sync)
and log, so a PROMPT.md hiccup can never brick task management. Added a symptom-
verification test that forces EISDIR and asserts getTask/updateTask/archiveTask
still succeed.

#10c: the dashboard badge-snapshot cache only evicted on hard-delete, so
archived tasks were re-cached via task:updated and retained for the daemon's
lifetime — a slow memory leak. New isBadgeEligibleTask predicate gates the
create/update listeners so archived tasks are evicted (matching the startup
prime's includeArchived:false). Added a unit test for the invariant.

Updates the #5 changeset to cover the real fix; adds a badge-eviction changeset.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
gsxdsm
2026-07-10 19:53:56 -07:00
parent 23e36b8935
commit 93b080102e
6 changed files with 222 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.changeset/badge-snapshot-archive-eviction.md Normal file
View File

@@ -0,0 +1,7 @@
---
"@runfusion/fusion": patch
---
summary: Fix a slow dashboard memory leak where archived tasks were never evicted from the in-memory badge cache.
category: performance
dev: The badge-snapshot cache (packages/dashboard/src/server.ts) only removed a task on hard-delete, so archiving a task re-cached it via the task:updated listener and it was retained for the daemon's lifetime — unbounded growth over long uptimes with task churn. A new `isBadgeEligibleTask` predicate (column !== "archived") gates both the create and update listeners so archived tasks are evicted, matching the startup prime's `includeArchived:false`. An unarchive re-primes the entry.

4
.changeset/task-api-500-diagnostics.md
View File

@@ -2,6 +2,6 @@
"@runfusion/fusion": patch
---
summary: Server now logs the underlying stack behind an API 500 so opaque task-endpoint failures are diagnosable.
summary: Task API operations no longer fail with 500 when a task's PROMPT.md can't be read; server also logs 500 causes.
category: fix
dev: `rethrowAsApiError` (packages/dashboard/src/api-error.ts) now preserves the original error as Error `cause` instead of discarding it, and `sendErrorResponse`/the `/api` error boundary (packages/dashboard/src/server.ts) log the stack + cause chain for 5xx (not just the message). The client-facing body stays generic in production. Unblocks root-causing the reported "task write API returns 500 for every task" (GET/DELETE/PATCH/retry/archive/reset on /api/tasks/:id) whose cause was previously never recorded server-side.
dev: getTask (the shared load for GET/DELETE/PATCH/retry/reset/archive) and the mutation helpers updateTaskUnlocked, updateStep, readPromptForArchive, and resetPromptCheckboxes (packages/core/src/store.ts) read PROMPT.md unguarded, so an unreadable file (root-owned from a prior `sudo` run → EACCES, PROMPT.md being a directory → EISDIR, transient FS error) 500'd every per-task op while the PROMPT.md-free board list/create kept working. These reads are now best-effort (degrade + log). Diagnosability: rethrowAsApiError preserves the original error as Error `cause` and the /api boundary logs stack + cause for 5xx (packages/dashboard/src/api-error.ts, server.ts); client body stays generic in production.

76
packages/core/src/__tests__/task-detail-prompt-resilience.test.ts Normal file
View File

@@ -0,0 +1,76 @@
/*
FNXC:TaskDetailPromptResilience 2026-07-10-15:00:
Symptom verification for the "task write API returns 500 for nearly everything" report:
GET/DELETE/PATCH/retry/reset/archive on a task all 500'd for every task (healthy ones
too) while the board list and create worked. Root cause: getTask — the shared load for
the entire per-task API — reads PROMPT.md directly and unguarded, so any read failure
(a root-owned PROMPT.md from a prior `sudo` run → EACCES, PROMPT.md being a directory →
EISDIR, etc.) threw and bricked every per-task operation, whereas the slim board list
never touches PROMPT.md.
Original symptom: getTask throws on an unreadable PROMPT.md, 500ing all per-task ops.
Exact reproduction: replace a task's PROMPT.md with a directory so readFile raises EISDIR.
Assertion it is gone: getTask resolves with the row (prompt degraded to ""), and a
representative mutation still succeeds — the per-task API is no longer bricked.
*/
import { afterEach, describe, expect, it } from "vitest";
import { existsSync, mkdirSync, mkdtempSync, rmSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
const cleanupDirs: string[] = [];
afterEach(() => {
for (const dir of cleanupDirs.splice(0)) {
rmSync(dir, { recursive: true, force: true });
}
});
describe("getTask PROMPT.md read resilience (task-write-API 500 regression)", () => {
it("returns the task detail (and mutations still work) when PROMPT.md cannot be read", async () => {
const { TaskStore } = await import("../store.js");
const root = mkdtempSync(join(tmpdir(), "fn-task-detail-resilience-"));
cleanupDirs.push(root);
const store = new TaskStore(root);
await store.init();
try {
const task = await store.createTask({ description: "task whose PROMPT.md becomes unreadable" });
// Baseline: a healthy task detail loads and carries the prompt text.
const baseline = await store.getTask(task.id);
expect(baseline.id).toBe(task.id);
expect(typeof baseline.prompt).toBe("string");
// Make PROMPT.md unreadable deterministically: replace the file with a
// directory so `readFile` raises EISDIR (mirrors the EACCES a root-owned
// PROMPT.md produces, without depending on chmod/uid semantics).
const promptPath = join(root, ".fusion", "tasks", task.id, "PROMPT.md");
rmSync(promptPath, { force: true });
mkdirSync(promptPath, { recursive: true });
expect(existsSync(promptPath)).toBe(true);
// The read path (GET /api/tasks/:id) must NOT throw — it degrades to an
// empty prompt instead of 500ing.
const detail = await store.getTask(task.id);
expect(detail.id).toBe(task.id);
expect(detail.prompt).toBe("");
// The mutation path must stay usable too. These store methods back the
// reported failing endpoints and each independently touches PROMPT.md:
// PATCH -> updateTask (title/description PROMPT.md heading sync)
// archive-> archiveTask (readPromptForArchive)
// A read failure in that PROMPT.md work must not brick the DB mutation.
await expect(store.updateTask(task.id, { title: "renamed with broken PROMPT.md" })).resolves.toBeTruthy();
const afterMutation = await store.getTask(task.id);
expect(afterMutation.title).toBe("renamed with broken PROMPT.md");
await expect(store.archiveTask(task.id)).resolves.toBeTruthy();
const archived = await store.getTask(task.id);
expect(archived.column).toBe("archived");
} finally {
await store.close();
}
});
});

123
packages/core/src/store.ts
View File

@@ -2436,7 +2436,15 @@ export class TaskStore extends EventEmitter<TaskStoreEvents> {
if (!existsSync(promptPath)) {
return undefined;
}
return readFile(promptPath, "utf-8");
// FNXC:TaskDetailPromptResilience 2026-07-10-15:00: best-effort — an
// unreadable PROMPT.md must not fail archiving (a reported failing per-task
// op); the archive entry simply omits the prompt text.
try {
return await readFile(promptPath, "utf-8");
} catch (err) {
storeLog.warn(`[task-detail] failed to read PROMPT.md for archive of ${taskId}: ${getErrorMessage(err)}`);
return undefined;
}
}
private async buildArchivedAgentLogFields(
@@ -5533,15 +5541,34 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS}
// Derived at read time only; retrySummary is never persisted to SQLite.
task.retrySummary = computeRetrySummary(task);
// Sync steps from PROMPT.md if task.steps is empty
/*
FNXC:TaskDetailPromptResilience 2026-07-10-15:00:
PROMPT.md is enrichment for the task detail (the `prompt` text and, when steps
are unpersisted, step-syncing) — NOT essential row data. getTask is the shared
load for the entire per-task API (GET/DELETE/PATCH/retry/reset/archive), so an
unguarded read/parse throw here turned every per-task operation into a 500 while
the PROMPT.md-free board list kept working — the reported "task write API returns
500 for every task". A read can fail for reasons unrelated to the row: a
root-owned PROMPT.md left by a prior `sudo` run (EACCES), PROMPT.md being a
directory (EISDIR), a symlink loop, or a transient FS error. Degrade to empty
prompt / unsynced steps and log, rather than bricking task management.
*/
if (task.steps.length === 0) {
task.steps = await this.parseStepsFromPrompt(id);
try {
task.steps = await this.parseStepsFromPrompt(id);
} catch (err) {
storeLog.warn(`[task-detail] failed to sync steps from PROMPT.md for ${id}: ${getErrorMessage(err)}`);
}
}
let prompt = "";
const promptPath = join(this.taskDir(id), "PROMPT.md");
if (existsSync(promptPath)) {
prompt = await readFile(promptPath, "utf-8");
try {
const promptPath = join(this.taskDir(id), "PROMPT.md");
if (existsSync(promptPath)) {
prompt = await readFile(promptPath, "utf-8");
}
} catch (err) {
storeLog.warn(`[task-detail] failed to read PROMPT.md for ${id}: ${getErrorMessage(err)}`);
}
return { ...task, prompt };
@@ -8340,11 +8367,18 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS}
return;
}
const content = await readFile(promptPath, "utf-8");
const resetContent = content.replace(/^- \[x\]/gm, "- [ ]");
// FNXC:TaskDetailPromptResilience 2026-07-10-15:00: cosmetic checkbox reset —
// an unreadable/unwritable PROMPT.md must not fail the task reset itself (a
// reported failing per-task op); the DB reset already proceeded.
try {
const content = await readFile(promptPath, "utf-8");
const resetContent = content.replace(/^- \[x\]/gm, "- [ ]");
if (resetContent !== content) {
await writeFile(promptPath, resetContent, "utf-8");
if (resetContent !== content) {
await writeFile(promptPath, resetContent, "utf-8");
}
} catch (err) {
storeLog.warn(`[task-detail] failed to reset PROMPT.md checkboxes in ${dir}: ${getErrorMessage(err)}`);
}
}
@@ -9650,34 +9684,46 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS}
// task.json remains the canonical source for title/description fields.
// PROMPT.md is only ever fully rewritten via explicit `updates.prompt`.
if (updates.prompt === undefined && (updates.title !== undefined || updates.description !== undefined)) {
// FNXC:TaskDetailPromptResilience 2026-07-10-15:00:
// Keeping the human-visible PROMPT.md heading/mission in sync with
// task.json is cosmetic — the DB row (persisted above) is canonical. An
// unreadable/unwritable PROMPT.md (root-owned from a prior `sudo` run →
// EACCES, PROMPT.md being a directory → EISDIR, transient FS error) must
// NOT fail the update itself, or every title/description edit 500s
// exactly like the reported task-write-API failure. Best-effort: log and
// skip the sync on failure.
const promptPath = join(dir, "PROMPT.md");
if (existsSync(promptPath)) {
const existingPrompt = await readFile(promptPath, "utf-8");
try {
if (existsSync(promptPath)) {
const existingPrompt = await readFile(promptPath, "utf-8");
if (isBootstrapPromptStub(existingPrompt, task.id, preUpdateTitle, preUpdateDescription)) {
const newPrompt = buildBootstrapPrompt(task.id, task.title, task.description);
await writeFile(promptPath, newPrompt);
} else {
// Real spec — surgical edits only. Each section we propagate to is
// edited in place; everything else (Review Level, Frontend UX
// Criteria, custom sections from triage) is preserved verbatim.
let next = existingPrompt;
if (updates.title !== undefined) {
// Match the existing heading style: triage emits
// `# Task: {id} - {title}`; createTask uses `# {id}: {title}`.
const triageStyle = /^#\s+Task:\s+[A-Z]+-\d+\s+-\s+/m.test(existingPrompt);
const heading = triageStyle
? (task.title ? `Task: ${task.id} - ${task.title}` : `Task: ${task.id}`)
: (task.title ? `${task.id}: ${task.title}` : task.id);
next = rewriteHeadingLine(next, heading);
}
if (updates.description !== undefined) {
next = rewriteMissionSection(next, task.description);
}
if (next !== existingPrompt) {
await writeFile(promptPath, next);
if (isBootstrapPromptStub(existingPrompt, task.id, preUpdateTitle, preUpdateDescription)) {
const newPrompt = buildBootstrapPrompt(task.id, task.title, task.description);
await writeFile(promptPath, newPrompt);
} else {
// Real spec — surgical edits only. Each section we propagate to is
// edited in place; everything else (Review Level, Frontend UX
// Criteria, custom sections from triage) is preserved verbatim.
let next = existingPrompt;
if (updates.title !== undefined) {
// Match the existing heading style: triage emits
// `# Task: {id} - {title}`; createTask uses `# {id}: {title}`.
const triageStyle = /^#\s+Task:\s+[A-Z]+-\d+\s+-\s+/m.test(existingPrompt);
const heading = triageStyle
? (task.title ? `Task: ${task.id} - ${task.title}` : `Task: ${task.id}`)
: (task.title ? `${task.id}: ${task.title}` : task.id);
next = rewriteHeadingLine(next, heading);
}
if (updates.description !== undefined) {
next = rewriteMissionSection(next, task.description);
}
if (next !== existingPrompt) {
await writeFile(promptPath, next);
}
}
}
} catch (err) {
storeLog.warn(`[task-detail] failed to sync PROMPT.md heading for ${task.id}: ${getErrorMessage(err)}`);
}
}
@@ -9900,8 +9946,15 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS}
// Auto-initialize steps from PROMPT.md if empty. Bypassed for graph-source
// writes (U6/KTD-3): the graph owns explicit indices pinned at expansion.
// FNXC:TaskDetailPromptResilience 2026-07-10-15:00: step auto-init is
// best-effort — an unreadable PROMPT.md must not fail updateStep (on the
// reported reset path); proceed with the persisted (empty) steps.
if (task.steps.length === 0 && !graphSource) {
task.steps = await this.parseStepsFromPrompt(id);
try {
task.steps = await this.parseStepsFromPrompt(id);
} catch (err) {
storeLog.warn(`[task-detail] failed to auto-init steps from PROMPT.md for ${id}: ${getErrorMessage(err)}`);
}
}
// Initialize log array if missing (for legacy tasks)

21
packages/dashboard/src/__tests__/badge-snapshot-eviction.test.ts Normal file
View File

@@ -0,0 +1,21 @@
import { describe, expect, it } from "vitest";
import { isBadgeEligibleTask } from "../server.js";
/*
FNXC:BadgeSnapshotEviction 2026-07-10-15:00:
Regression for the slow badge-cache memory leak: archived tasks were re-cached on
task:updated but only evicted on hard-delete, so they accumulated for the daemon's
lifetime. The badge cache is board-scoped — archived tasks must be ineligible so both
the create and update listeners evict rather than retain them.
*/
describe("isBadgeEligibleTask", () => {
it("excludes archived tasks from the live-board badge cache", () => {
expect(isBadgeEligibleTask({ column: "archived" })).toBe(false);
});
it("includes tasks on any live board column", () => {
for (const column of ["todo", "in-progress", "in-review", "done"] as const) {
expect(isBadgeEligibleTask({ column })).toBe(true);
}
});
});

28
packages/dashboard/src/server.ts
View File

@@ -2445,6 +2445,14 @@ export function setupBadgeWebSocket(
const onTaskUpdated = (task: Task) => {
const cacheKey = `${scopeKey}:${task.id}`;
// FNXC:BadgeSnapshotEviction 2026-07-10-15:00: evict (not re-cache) when a
// task is archived off the live board, and skip the publish so peers don't
// re-cache it. An unarchive re-emits task:updated with a live column and
// re-primes the entry. See isBadgeEligibleTask.
if (!isBadgeEligibleTask(task)) {
badgeSnapshots.delete(cacheKey);
return;
}
const previousSnapshot = badgeSnapshots.get(cacheKey);
const nextSnapshot: BadgeSnapshot = {
prInfo: task.prInfo ?? null,
@@ -2480,6 +2488,13 @@ export function setupBadgeWebSocket(
const onTaskCreated = (task: Task) => {
const cacheKey = `${scopeKey}:${task.id}`;
// FNXC:BadgeSnapshotEviction 2026-07-10-15:00: an already-archived task
// (e.g. restored/imported into the archive) must not seed the live-board
// badge cache — same eligibility rule as the update listener.
if (!isBadgeEligibleTask(task)) {
badgeSnapshots.delete(cacheKey);
return;
}
badgeSnapshots.set(cacheKey, {
prInfo: task.prInfo ?? null,
issueInfo: task.issueInfo ?? null,
@@ -2603,6 +2618,19 @@ export function setupBadgeWebSocket(
});
}
/*
FNXC:BadgeSnapshotEviction 2026-07-10-15:00:
The in-memory badge-snapshot cache is keyed by task id and only ever removed a task
on hard-delete, so archived tasks accumulated for the daemon's whole lifetime — a slow
memory leak on long-running servers with task churn. Badge snapshots are only needed for
tasks visible on the live board; archived tasks leave it. This predicate is the single
eligibility rule used by both the create and update listeners (and mirrored by the
startup prime's `includeArchived:false`). Exported for unit coverage of the invariant.
*/
export function isBadgeEligibleTask(task: Pick<Task, "column">): boolean {
return task.column !== "archived";
}
/** Compare two badge snapshots for equality */
function snapshotsEqual(a: BadgeSnapshot | undefined, b: BadgeSnapshot | undefined): boolean {
if (!a && !b) return true;