root/fusion
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

FN-8174: preserve live triage planning sessions

Browse Source 
Keep active planning sessions protected from stale recovery while reclaiming genuinely hung triage work.

- Retain stale processing entries that still have a live, non-aborted triage session.
- Continue evicting no-session and stuck-aborted tasks so recovery can proceed.
- Add triage and self-healing regression coverage, architecture guidance, and a patch changeset.

Files changed:
 .changeset/fn-8174-planning-premature-todo.md      |   7 ++
 docs/architecture.md                               |   1 +
 packages/engine/src/__tests__/self-healing.test.ts | 102 +++++++++++++++++++++
 packages/engine/src/__tests__/triage.test.ts       |  37 +++++++-
 packages/engine/src/triage.ts                      |  48 +++++-----
 5 files changed, 168 insertions(+), 27 deletions(-)

Fusion-Task-Id: FN-8174

Fusion-Task-Lineage: f6811d72-95b4-4b5f-a71f-212f50e3ecdd

Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
gsxdsm
2026-07-16 18:54:19 -07:00
parent eab9977a42
commit b687cc994e
5 changed files with 168 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.changeset/fn-8174-planning-premature-todo.md Normal file
View File

@@ -0,0 +1,7 @@
---
"@runfusion/fusion": patch
---
summary: Stop tasks that are still being planned from being moved to Todo prematurely.
category: fix
dev: Stale triage eviction retains live non-aborted sessions while reclaiming stuck-aborted and no-session hangs.

1
docs/architecture.md
View File

@@ -972,6 +972,7 @@ Key server capabilities:
- Refinements still require normal triage specification (PROMPT.md with valid `File Scope`) before execution routing.
- To prevent starvation under large same-priority planning backlogs (FN-4647 pattern), triage polling now prefers `task_refine` rows over non-refinement rows as an ordering tiebreaker within the same priority band.
- **Starved refinement self-healing sweep (Lane B):** `SelfHealingManager.recoverStarvedRefinementTriageTasks()` runs in startup + maintenance sweeps and targets `sourceType: "task_refine"` tasks still in `triage` (`status` `null|planning`) that are unpaused, not actively planning, older than `STARVED_REFINEMENT_RECOVERY_GRACE_MS` (10m), and have observed peer board progress (`STARVED_PEER_PROGRESS_THRESHOLD=3` non-refinement tasks advanced to `todo` after the refinement was created). Remediation is a bounded one-step priority nudge (no direct move-to-`todo`) with cooldown idempotency (`STARVED_REFINEMENT_ESCALATION_COOLDOWN_MS = grace*4`) and run-audit emission `task:auto-recover-starved-refinement` including `{ taskId, ageMs, peerProgressCount, escalation }` metadata.
- **Stale planning liveness invariant:** stale-processing eviction never removes a task with a live, non-aborted triage session (`activeSessions.has(id) && !stuckAborted.has(id)`). It therefore remains in `getProcessingTaskIds()` and cannot be finalized, cleared for replanning, or priority-nudged by planning recovery sweeps. Hung promises without a session and stuck-aborted/disposed sessions remain reclaimable after the stale threshold.
- Approval semantics are unchanged: with `requirePlanApproval=true`, refinements stop at `status: "awaiting-approval"`; otherwise they move to `todo` after spec finalization.
- Regression coverage lives in `packages/engine/src/__tests__/triage-refinement-routing.test.ts` and locks four guarantees: bounded promotion under backlog pressure, approval-gate preservation, PROMPT-before-`todo` invariant, and unchanged baseline ordering for non-refinement-only triage sets.
- Task detail surface is shared through `TaskDetailContent` (exported from `TaskDetailModal.tsx`): desktop/tablet `ListView` renders it inline in the split right pane, while mobile and non-list entry points continue using `TaskDetailModal`.

102
packages/engine/src/__tests__/self-healing.test.ts
View File

@@ -8878,6 +8878,108 @@ describe("stale triage processing eviction before recovery", () => {
manager.stop();
});
it("preserves live planning tasks across approved, orphaned, and starved recovery sweeps", async () => {
const store = createMockStore();
const liveIds = new Set(["FN-approved-live", "FN-orphan-live", "FN-refinement-live"]);
const evictFn = vi.fn().mockReturnValue(new Set<string>());
const recoverFn = vi.fn().mockResolvedValue(true);
const manager = new SelfHealingManager(store, {
rootDir: "/tmp/test-project",
recoverApprovedTriageTask: recoverFn,
getPlanningTaskIds: () => liveIds,
evictStaleTriageProcessing: evictFn,
});
const old = "2026-01-01T00:00:00.000Z";
(store.listTasks as ReturnType<typeof vi.fn>).mockResolvedValue([
{ id: "FN-approved-live", column: "triage", status: "planning", paused: false, priority: "normal", createdAt: old, updatedAt: old },
{ id: "FN-orphan-live", column: "triage", status: "planning", paused: false, priority: "normal", createdAt: old, updatedAt: old },
{ id: "FN-refinement-live", column: "triage", status: "planning", paused: false, priority: "normal", sourceType: "task_refine", createdAt: old, updatedAt: old },
{ id: "FN-peer-1", column: "todo", sourceType: "dashboard_ui", createdAt: old, updatedAt: "2026-01-01T00:01:00.000Z" },
{ id: "FN-peer-2", column: "todo", sourceType: "dashboard_ui", createdAt: old, updatedAt: "2026-01-01T00:02:00.000Z" },
{ id: "FN-peer-3", column: "todo", sourceType: "dashboard_ui", createdAt: old, updatedAt: "2026-01-01T00:03:00.000Z" },
]);
vi.setSystemTime(new Date("2026-01-01T01:00:00.000Z"));
expect(await manager.recoverApprovedTriageTasks()).toBe(0);
expect(await manager.recoverOrphanedPlanningTasks()).toBe(0);
expect(await manager.recoverStarvedRefinementTriageTasks()).toBe(0);
expect(evictFn).toHaveBeenCalledTimes(3);
expect(recoverFn).not.toHaveBeenCalled();
expect(store.updateTask).not.toHaveBeenCalled();
manager.stop();
});
it("recovers no-session and stuck-aborted planning IDs after eviction removes both", async () => {
const store = createMockStore();
let planningIds = new Set(["FN-live", "FN-hung", "FN-stuck-aborted"]);
const evictFn = vi.fn().mockImplementation(() => {
planningIds = new Set(["FN-live"]);
return new Set(["FN-hung", "FN-stuck-aborted"]);
});
const recoverFn = vi.fn().mockResolvedValue(true);
const manager = new SelfHealingManager(store, {
rootDir: "/tmp/test-project",
recoverApprovedTriageTask: recoverFn,
getPlanningTaskIds: () => planningIds,
evictStaleTriageProcessing: evictFn,
});
const old = "2026-01-01T00:00:00.000Z";
(store.listTasks as ReturnType<typeof vi.fn>).mockResolvedValue([
{ id: "FN-live", column: "triage", status: "planning", paused: false, priority: "normal", createdAt: old, updatedAt: old },
{ id: "FN-hung", column: "triage", status: "planning", paused: false, priority: "normal", createdAt: old, updatedAt: old },
{ id: "FN-stuck-aborted", column: "triage", status: "planning", paused: false, priority: "normal", createdAt: old, updatedAt: old },
]);
vi.setSystemTime(new Date("2026-01-01T01:00:00.000Z"));
expect(await manager.recoverApprovedTriageTasks()).toBe(2);
expect(recoverFn).toHaveBeenCalledWith(expect.objectContaining({ id: "FN-hung" }));
expect(recoverFn).toHaveBeenCalledWith(expect.objectContaining({ id: "FN-stuck-aborted" }));
expect(recoverFn).not.toHaveBeenCalledWith(expect.objectContaining({ id: "FN-live" }));
manager.stop();
});
it("clears and priority-nudges evicted hung and stuck-aborted tasks while preserving live planning", async () => {
const store = createMockStore();
const evictFn = vi.fn().mockReturnValue(new Set(["FN-hung", "FN-stuck-aborted"]));
const getPlanning = vi.fn().mockReturnValue(new Set(["FN-live"]));
const manager = new SelfHealingManager(store, {
rootDir: "/tmp/test-project",
getPlanningTaskIds: getPlanning,
evictStaleTriageProcessing: evictFn,
});
const old = "2026-01-01T00:00:00.000Z";
const planningTasks = [
{ id: "FN-live", column: "triage", status: "planning", paused: false, priority: "normal", createdAt: old, updatedAt: old },
{ id: "FN-hung", column: "triage", status: "planning", paused: false, priority: "normal", createdAt: old, updatedAt: old },
{ id: "FN-stuck-aborted", column: "triage", status: "planning", paused: false, priority: "normal", createdAt: old, updatedAt: old },
];
(store.listTasks as ReturnType<typeof vi.fn>).mockResolvedValue(planningTasks);
vi.setSystemTime(new Date("2026-01-01T01:00:00.000Z"));
expect(await manager.recoverOrphanedPlanningTasks()).toBe(2);
expect(store.updateTask).toHaveBeenCalledWith("FN-hung", { status: null });
expect(store.updateTask).toHaveBeenCalledWith("FN-stuck-aborted", { status: null });
expect(store.updateTask).not.toHaveBeenCalledWith("FN-live", { status: null });
vi.clearAllMocks();
(store.listTasks as ReturnType<typeof vi.fn>).mockResolvedValue([
...planningTasks.map((task) => ({ ...task, sourceType: "task_refine" })),
{ id: "FN-peer-1", column: "todo", sourceType: "dashboard_ui", createdAt: old, updatedAt: "2026-01-01T00:01:00.000Z" },
{ id: "FN-peer-2", column: "todo", sourceType: "dashboard_ui", createdAt: old, updatedAt: "2026-01-01T00:02:00.000Z" },
{ id: "FN-peer-3", column: "todo", sourceType: "dashboard_ui", createdAt: old, updatedAt: "2026-01-01T00:03:00.000Z" },
]);
expect(await manager.recoverStarvedRefinementTriageTasks()).toBe(2);
expect(store.updateTask).toHaveBeenCalledWith("FN-hung", { priority: "high" });
expect(store.updateTask).toHaveBeenCalledWith("FN-stuck-aborted", { priority: "high" });
expect(store.updateTask).not.toHaveBeenCalledWith("FN-live", { priority: "high" });
manager.stop();
});
});
// ── Maintenance cycle concurrency ──────────────────────────────────

37
packages/engine/src/__tests__/triage.test.ts
View File

@@ -6955,21 +6955,48 @@ describe("evictStaleProcessing", () => {
expect(processor.getProcessingTaskIds().has("FN-001")).toBe(false);
});
it("does not evict tasks that have been in processing less than 30 minutes", () => {
it("retains stale tasks with a live non-aborted triage session", () => {
const store = createMockStore();
const processor = new TriageProcessor(store, "/tmp/root");
vi.setSystemTime(new Date("2026-01-01T00:00:00.000Z"));
(processor as any).processing.add("FN-001");
(processor as any).processingSince.set("FN-001", Date.now());
(processor as any).processing.add("FN-live");
(processor as any).processingSince.set("FN-live", Date.now());
(processor as any).activeSessions.set("FN-live", { dispose: vi.fn() });
// Advance time 29 minutes — not stale yet
vi.setSystemTime(new Date("2026-01-01T00:31:00.000Z"));
const evicted = processor.evictStaleProcessing();
expect(evicted).toEqual(new Set());
expect(processor.getProcessingTaskIds().has("FN-live")).toBe(true);
expect((processor as any).activeSessions.has("FN-live")).toBe(true);
});
it("does not evict tasks that have been in processing less than 30 minutes regardless of session state", () => {
const store = createMockStore();
const processor = new TriageProcessor(store, "/tmp/root");
vi.setSystemTime(new Date("2026-01-01T00:00:00.000Z"));
for (const taskId of ["FN-no-session", "FN-live", "FN-stuck-aborted"]) {
(processor as any).processing.add(taskId);
(processor as any).processingSince.set(taskId, Date.now());
}
(processor as any).activeSessions.set("FN-live", { dispose: vi.fn() });
(processor as any).activeSessions.set("FN-stuck-aborted", { dispose: vi.fn() });
(processor as any).stuckAborted.add("FN-stuck-aborted");
// Advance time 29 minutes — not stale yet.
vi.setSystemTime(new Date("2026-01-01T00:29:00.000Z"));
const evicted = processor.evictStaleProcessing();
expect(evicted.size).toBe(0);
expect(processor.getProcessingTaskIds().has("FN-001")).toBe(true);
expect(processor.getProcessingTaskIds()).toEqual(new Set([
"FN-no-session",
"FN-live",
"FN-stuck-aborted",
]));
});
it("cleans up activeSessions and stuckAborted when evicting", () => {

48
packages/engine/src/triage.ts
View File

@@ -537,21 +537,18 @@ export class TriageProcessor {
}
/**
* Maximum time a task can remain in the `processing` set before it's
* considered stale (30 minutes). By this point the stuck detector
* (default 20-min timeout) should have already killed the session
* and the `finally` block should have cleaned up. If it hasn't,
* the promise is hung (e.g., `promptWithFallback` never settled
* after dispose) and self-healing recovery needs to force-evict it.
* Maximum time a task can remain in the `processing` set before a hung,
* non-live session is considered stale (30 minutes). A live session remains
* protected regardless of elapsed time; a stuck-aborted session is still
* reclaimable because its promise may never reach the cleanup `finally`.
*/
private static readonly STALE_PROCESSING_THRESHOLD_MS = 30 * 60 * 1000;
/**
* Evict tasks from the `processing` set that have been there longer than
* the staleness threshold. This handles the case where a stuck-kill
* disposes the session but the `specifyTask` promise never settles
* (hung `promptWithFallback`), leaving the task in `processing` forever
* and blocking self-healing recovery.
* Evict stale tasks from `processing` only when their triage promise is no
* longer live. This reclaims a stuck-killed/disposed session whose
* `specifyTask` promise never settles, while preserving a session still
* streaming past the normal wall-clock threshold.
*
* @returns the set of evicted task IDs
*/
@@ -561,17 +558,24 @@ export class TriageProcessor {
const evicted = new Set<string>();
for (const [taskId, since] of this.processingSince) {
if (now - since >= threshold) {
planLog.warn(
`${taskId} has been in processing for ${Math.round((now - since) / 60_000)}min ` +
`(threshold: ${Math.round(threshold / 60_000)}min) — evicting (likely hung promise)`,
);
this.processing.delete(taskId);
this.processingSince.delete(taskId);
this.activeSessions.delete(taskId);
this.stuckAborted.delete(taskId);
evicted.add(taskId);
}
if (now - since < threshold) continue;
/*
FNXC:Triage 2026-07-16-18:29:
Stale-processing eviction must retain a task with a live, non-aborted triage session (`activeSessions.has(id) && !stuckAborted.has(id)`). Removing it would drop genuinely active planning from `getProcessingTaskIds()` and let self-healing prematurely finalize it to todo/awaiting-approval, clear planning status, or nudge priority. Hung promises without a session and stuck-aborted/disposed sessions remain evictable.
*/
const hasLiveSession = this.activeSessions.has(taskId) && !this.stuckAborted.has(taskId);
if (hasLiveSession) continue;
planLog.warn(
`${taskId} has been in processing for ${Math.round((now - since) / 60_000)}min ` +
`(threshold: ${Math.round(threshold / 60_000)}min) — evicting (likely hung promise)`,
);
this.processing.delete(taskId);
this.processingSince.delete(taskId);
this.activeSessions.delete(taskId);
this.stuckAborted.delete(taskId);
evicted.add(taskId);
}
return evicted;