fix(engine): close executor/merger concurrency races and reviewer pause TOCTOU
FN-2910 surfaced concurrent reviewer + merger activity on the same task. Root cause: asymmetric in-flight guards let an unpause-resume kick off a fresh executor session while a recovery path was already running, and the auto-merge handoff fired before the executor's finally block finished cleanup. This sweeps the surrounding lifecycle paths for similar races and tightens the reviewer pause gate against TOCTOU through runtime setup. - Symmetric in-flight tracking across `executing`, `recoveringCompleted`, and `resumingUnpaused`; `recoverCompletedTask` bails when any are set. - Atomic claim of the recovery slot in the completed-task watchdog before any awaited work. - Workflow-rerun bounce returns "bounced" | "skipped-pending" so the watchdog can no longer log a false-success retry when the original bounce is still mid-flight. - Self-healing's completed-task scan re-checks executing IDs inside the loop instead of trusting a pre-await snapshot. - 300ms grace period before auto-merge enqueue, giving the executor's finally block (session disposal, child cleanup) time to drain and eliminating the residual log-overlap symptom from FN-2910. Test uses fake timers, no real sleep added. - New AgentSemaphore.runNested for synchronously nested helper agents (reviewers): bumps activeCount for honest observability while bypassing the wait queue, preserving forward-progress fairness for the parent at low maxConcurrent. Both createReviewStepTool and triage's createReviewSpecTool now use it. - New beforeSpawnSession hook on AgentRuntimeOptions/AgentOptions fired inside createFnAgent immediately before createAgentSession, past every awaited setup step. Reviewer wires a pause re-check that throws a sentinel error converted to UNAVAILABLE, closing the TOCTOU window where pause flipped during runtime resolution or resource loading. All 2887 engine tests pass; engine + core + cli + dashboard + plugin-sdk + pi-claude-cli + desktop typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,7 +1,6 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { ProjectEngine } from "../project-engine.js";
|
||||
import { runtimeLog } from "../logger.js";
|
||||
import { aiMergeTask } from "../merger.js";
|
||||
import { TunnelProcessManager } from "../remote-access/tunnel-process-manager.js";
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
@@ -1224,11 +1223,18 @@ describe("ProjectEngine swallowed error hardening", () => {
|
||||
expect(handler).toBeTypeOf("function");
|
||||
if (!handler) throw new Error("task:moved handler was not registered");
|
||||
|
||||
// Auto-merge enqueue runs inside a setTimeout grace period (~300ms) to
|
||||
// let the executor's finally block complete before the merger starts.
|
||||
// Use fake timers so the test doesn't actually sleep 300ms.
|
||||
vi.useFakeTimers();
|
||||
|
||||
await handler({
|
||||
task: { id: "FN-001", column: "in-review" },
|
||||
to: "in-review",
|
||||
});
|
||||
|
||||
await vi.advanceTimersByTimeAsync(500);
|
||||
|
||||
expect(warnSpy).toHaveBeenCalledWith(
|
||||
expect.stringContaining("Auto-merge: failed to read settings for task:moved on FN-001"),
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user