8bef30655d
TaskExecutor.execute() had a classic JS async race window. Original:
async execute(task) {
if (this.executing.has(task.id)) return; // check
const assignedAgentId = task.assignedAgentId;
if (assignedAgentId && await this.shouldDeferForHeartbeat(...)) // AWAIT yields
return;
this.executing.add(task.id); // add (too late)
...
}
Two concurrent execute(task) calls (scheduler dispatch + task:moved event
handler + restart-recovery) both:
1. Pass the synchronous has() check (Set is empty).
2. Enter the awaited shouldDeferForHeartbeat call (yields the event loop).
3. Resume and both call this.executing.add(task.id).
4. Both proceed to create the same worktree path.
Production failure shape (FN-4814 + FN-4811, observed within minutes):
01:30:56 [runA-caoe] Worktree created at /...worktrees/bright-mesa
01:30:56 [runB-w23q] Worktree created at /...worktrees/bright-mesa
01:30:58 worktree liveness assertion failed: not_usable_task_worktree
01:31:48 [thirdRun] also fires liveness assertion fail
01:37:48 In-review stall surfaced [no-worktree-no-merge-confirmed]
This is the root cause of the entire FN-4781/FN-4804/FN-4814/FN-4811
cascade. Every other guard added today (FN-4811 active-session gate,
self-healing reclaim defer, validation-failed recovery, silent reclaim
recovery, integrity-warning dedup) was patching SYMPTOMS of the
duplicate-run race. With this fix, the symptoms stop appearing.
Fix: claim the slot synchronously immediately after the has() check,
release it on the heartbeat-defer early-return path. No await happens
between check and claim, so the race window is closed.
Test added under
packages/engine/src/__tests__/reliability-interactions/concurrent-execute-race.test.ts
verified to fail on the prior (a1b1f9aa0) executor.ts and pass on the
fixed version:
- Two concurrent execute() calls produce the SAME number of
createFnAgent invocations as one execute() call (no amplification).
- A second sequential execute() after the first completes IS allowed
(slot was released).
The task must have assignedAgentId set to exercise the race \u2014 without
it, the short-circuit `assignedAgentId && ...` evaluates the left side
to false synchronously, and no await happens.
Full engine suite: 5048+ tests pass. The 7 transient test-file failures
in the broad parallel run are pre-existing flaky real-git tests
(branch-conflicts-zero-unique, branch-conflicts-recovery,
merger-overlap-guard subprocess-guard contention) \u2014 all of them pass
when run alone or as a smaller group, none touch the executor.execute()
path.
Fusion-Task-Id: FN-4811
1.3 KiB
@runfusion/fusion
| @runfusion/fusion |
|---|
| patch |
fix(FN-4811): close concurrent-execute race that produced parallel runs for the same task
TaskExecutor.execute() had an async race: after the synchronous this.executing.has(task.id) check, the code awaited shouldDeferForHeartbeat(...) BEFORE adding to the executing Set. Two concurrent execute() calls (scheduler dispatch + task:moved listener + restart-recovery) could both pass the check, both yield on the await, then both add to the Set and both proceed to create the same worktree.
Production signature (FN-4814, FN-4811):
01:30:56 [runA-caoe] Worktree created at /Users/eclipxe/Projects/kb/.worktrees/bright-mesa
01:30:56 [runB-w23q] Worktree created at /Users/eclipxe/Projects/kb/.worktrees/bright-mesa
01:30:58 worktree liveness assertion failed: not_usable_task_worktree
This is the canonical source of FN-4781/FN-4804/FN-4814/FN-4811 mid-task worktree disappearance and cross-task contamination — every other guard in the stack (FN-4811 active-session gate, self-healing reclaim defer, etc.) was patching the symptoms of the duplicate-run race.
Fix: claim the executing slot synchronously immediately after the has() check, release it on the heartbeat-defer early return. Closes the race window entirely.