fix(FN-7965): let the overseer see executor-stage failures

`deriveSignalAndSources`'s executor branch never read `task.status`, so a row
parked `status: "failed"` — e.g. the terminal fn_task_done refusal/invariant
park — reported `signal: "progressing"` with the reason "Task is actively
executing in-progress work". The overseer observed a dead task as healthy and
took no action. `failed` was only ever derived for the merger/pull-request
stages, so the sole backstop was the FN-7743 2h stall proxy firing hours later.
This is exactly what FN-7965's audit trail shows: every intervention on a
terminally-parked task was action="observe", reason="Task is actively executing
in-progress work".

Report `failed` so recovery engages on the next poll. This adds no new policy:
a failed executor observation already routes to `retry_step` (executor sources
are `agent-log`, never an ERROR_SOURCE_KIND), bounded by
PLANNER_RECOVERY_MAX_ATTEMPTS and escalated on exhaustion.

Precedence and dedup preserved: `paused` still wins, so an operator/user-paused
row stays `blocked` and is never routed into autonomous recovery; and the reason
is a constant (never interpolating task.error/status) so the FN-7577
`stage|signal|reason` feed dedup still suppresses repeat observations.

Verified: the repro test fails with the branch disabled; paused-precedence,
healthy-card (FN-7577) and dedup guards added; overseer/recovery surfaces
93 passed + core planner-recovery 20/20; engine + dashboard typecheck clean;
`pnpm test:gate` green (294+122+63).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
gsxdsm
2026-07-15 16:32:17 -07:00
parent 214af98591
commit 0e84731d8a
3 changed files with 93 additions and 0 deletions

View File

@@ -0,0 +1,7 @@
---
"@runfusion/fusion": patch
---
summary: The planner overseer now notices a failed in-progress task immediately instead of after two hours.
category: fix
dev: FN-7965. `deriveSignalAndSources`'s `executor` branch never read `task.status`, so a row parked `status: "failed"` (e.g. the terminal fn_task_done refusal/invariant park) reported `signal: "progressing"` with reason "Task is actively executing in-progress work". `failed` was only ever derived for the merger/pull-request stages, leaving the FN-7743 2h stall proxy (`columnMovedAt ?? updatedAt`) as the sole backstop. The branch now reports `failed`, which routes into the pre-existing failed-signal policy — `retry_step` at executor stage (sources are `agent-log`, never an ERROR_SOURCE_KIND), bounded by `PLANNER_RECOVERY_MAX_ATTEMPTS` and escalated on exhaustion. `paused` keeps precedence (operator/user-paused stays `blocked`), the reason is held constant so the FN-7577 `stage|signal|reason` feed dedup still holds, and `status` was added to `OverseerTaskRef`.

View File

@@ -464,3 +464,73 @@ describe("PlannerOverseerMonitor.observeTask — FN-7743 executor stall detectio
expect(store.logEntry).toHaveBeenCalledTimes(2);
});
});
// FNXC:PlannerOversight 2026-07-15-17:05:
// FN-7965: a task parked `status: "failed"` at the executor stage reported
// `progressing` ("Task is actively executing in-progress work") because the
// derivation never read `status` — so the overseer treated a dead task as healthy
// and only reacted once the FN-7743 stall proxy fired hours later. `failed` had
// been derived for the merger/pull-request stages only. These tests lock the
// invariant across the enumerated surfaces: failed (failed), failed-but-paused
// (blocked — a human owns it), healthy (progressing, unchanged), and the
// terminal columns that must stay unmonitored.
describe("PlannerOverseerMonitor.observeTask — FN-7965 executor failure detection", () => {
it("reports failed for a non-paused in-progress task parked status=failed", async () => {
const monitor = new PlannerOverseerMonitor();
const task = taskFixture({
column: "in-progress",
status: "failed",
updatedAt: new Date().toISOString(),
columnMovedAt: new Date().toISOString(),
});
const observation = await monitor.observeTask(task, "autonomous");
// Must NOT wait for the 2h stall proxy: the failure is visible immediately.
expect(observation?.stage).toBe("executor");
expect(observation?.signal).toBe("failed");
});
it("keeps paused precedence: a paused failed task is blocked, not failed", async () => {
// A human owns an operator/user-paused row — it must never be routed into
// autonomous bounded recovery just because status is failed.
const monitor = new PlannerOverseerMonitor();
const task = taskFixture({
column: "in-progress",
status: "failed",
paused: true,
pausedReason: "error-unrecoverable",
});
const observation = await monitor.observeTask(task, "autonomous");
expect(observation?.signal).toBe("blocked");
});
it.each([undefined, null, "queued"])("still reports progressing for a healthy in-progress task (status=%s)", async (status) => {
// Negative control: FN-7577 — a healthy card must never flip to a problem
// signal, or every in-progress task shows the "recovering" badge.
const monitor = new PlannerOverseerMonitor();
const task = taskFixture({
column: "in-progress",
status: status as never,
updatedAt: new Date().toISOString(),
columnMovedAt: new Date().toISOString(),
});
const observation = await monitor.observeTask(task, "autonomous");
expect(observation?.signal).toBe("progressing");
});
it("keeps the failed reason constant so the FN-7577 feed dedup holds", async () => {
// The reason must not interpolate task.error/status, or an observation would
// be re-emitted every poll for the same unchanged failure.
const monitor = new PlannerOverseerMonitor();
const a = await monitor.observeTask(taskFixture({ id: "FN-1000", status: "failed" }), "autonomous");
const b = await monitor.observeTask(taskFixture({ id: "FN-1000", status: "failed" }), "autonomous");
expect(a?.reason).toBe(b?.reason);
expect(a?.reason).not.toMatch(/failed:|error/i);
});
});

View File

@@ -55,6 +55,7 @@ export type OverseerTaskRef = Pick<
Task,
| "id"
| "column"
| "status"
| "prInfo"
| "reviewState"
| "paused"
@@ -180,6 +181,21 @@ function deriveSignalAndSources(
};
}
/*
FNXC:PlannerOversight 2026-07-15-17:05:
FN-7965: an executor-stage row parked `status: "failed"` (e.g. the terminal fn_task_done refusal/invariant park) reported `progressing` — "Task is actively executing in-progress work" — because nothing here read `status`. The overseer observed a dead task as healthy and took no action; `failed` was only ever derived for the merger/pull-request stages, so the sole backstop was the FN-7743 stall proxy below firing HOURS later. Read the status so bounded recovery engages on the next poll instead.
`paused` deliberately keeps precedence above: an operator/user-paused row is `blocked` because a human owns it, not an autonomously recoverable failure.
Downstream this yields `retry_step` (executor sources are `agent-log`, never an ERROR_SOURCE_KIND), bounded by `PLANNER_RECOVERY_MAX_ATTEMPTS` and then escalated on exhaustion — the pre-existing failed-signal policy, not a new one.
The reason must stay CONSTANT per (stage|signal): the FN-7577 feed dedup keys on `stage|signal|reason`, so never interpolate `task.error`/`status` here — a per-failure string would re-emit an observation every poll.
*/
if (task.status === "failed") {
return {
signal: "failed",
reason: "Executor stage parked failed with work incomplete",
sources: [{ kind: "agent-log", ref: taskId }],
};
}
// FNXC:PlannerOversight 2026-07-09-00:00:
// FN-7743: a non-paused in-progress task whose executor session has gone
// silent (dead/hung agent, no commits/heartbeat) was previously ALWAYS