fix(engine): close executor/merger concurrency races and reviewer pause TOCTOU

FN-2910 surfaced concurrent reviewer + merger activity on the same task.
Root cause: asymmetric in-flight guards let an unpause-resume kick off a
fresh executor session while a recovery path was already running, and the
auto-merge handoff fired before the executor's finally block finished
cleanup. This sweeps the surrounding lifecycle paths for similar races and
tightens the reviewer pause gate against TOCTOU through runtime setup.

- Symmetric in-flight tracking across `executing`, `recoveringCompleted`,
  and `resumingUnpaused`; `recoverCompletedTask` bails when any are set.
- Atomic claim of the recovery slot in the completed-task watchdog before
  any awaited work.
- Workflow-rerun bounce returns "bounced" | "skipped-pending" so the
  watchdog can no longer log a false-success retry when the original
  bounce is still mid-flight.
- Self-healing's completed-task scan re-checks executing IDs inside the
  loop instead of trusting a pre-await snapshot.
- 300ms grace period before auto-merge enqueue, giving the executor's
  finally block (session disposal, child cleanup) time to drain and
  eliminating the residual log-overlap symptom from FN-2910. Test uses
  fake timers, no real sleep added.
- New AgentSemaphore.runNested for synchronously nested helper agents
  (reviewers): bumps activeCount for honest observability while bypassing
  the wait queue, preserving forward-progress fairness for the parent at
  low maxConcurrent. Both createReviewStepTool and triage's
  createReviewSpecTool now use it.
- New beforeSpawnSession hook on AgentRuntimeOptions/AgentOptions fired
  inside createFnAgent immediately before createAgentSession, past every
  awaited setup step. Reviewer wires a pause re-check that throws a
  sentinel error converted to UNAVAILABLE, closing the TOCTOU window
  where pause flipped during runtime resolution or resource loading.

All 2887 engine tests pass; engine + core + cli + dashboard + plugin-sdk
+ pi-claude-cli + desktop typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
gsxdsm
2026-04-29 11:26:39 -07:00
parent c6c89a0fdf
commit f1c87c38d4
10 changed files with 342 additions and 90 deletions

View File

@@ -27,6 +27,15 @@ export interface ResolvedSessionOptions extends AgentRuntimeOptions {
pluginRunner?: PluginRunner;
/** Optional runtime hint from task/agent configuration */
runtimeHint?: string;
/**
* `beforeSpawnSession` is inherited from {@link AgentRuntimeOptions} — see
* its definition there for the contract. Callers (e.g. the reviewer's
* pause gate) throw from this callback to cancel session creation when
* external state changed during the async setup window. Forwarded
* verbatim to `runtime.createSession()`; the runtime is responsible for
* invoking it at its latest synchronous point before the underlying LLM
* session is instantiated.
*/
}
/**
@@ -87,7 +96,10 @@ export async function createResolvedAgentSession(
`[${sessionPurpose}] Using runtime "${resolved.runtimeId}" (configured=${resolved.wasConfigured})`,
);
// Create the session using the resolved runtime
// Forward `beforeSpawnSession` to the runtime so it fires at the true
// latest sync point (just before LLM session instantiation) rather than
// here, before the runtime's own awaited setup work runs. See
// AgentRuntimeOptions.beforeSpawnSession for the contract.
const result = await resolved.runtime.createSession(runtimeOptions);
// Attach the resolved runtime's promptWithFallback as a bound method on the