b6df11a6a8
Investigating FN-4814 + FN-4811 re-failures after commit8bef30655(which added per-instance synchronous this.executing.add) revealed the per-instance guard was insufficient. FN-4809 log at 02:48:17-18 UTC: 02:48:17 [-] Resuming execution after unpause 02:48:17 [-] Step 4 (Testing & Verification) -> pending 02:48:17 [-] Step 4 (Testing & Verification) -> pending 02:48:17 [6097725-y2nb] Executor detected stale merge state ... 02:48:18 [6097816-9gde] Executor detected stale merge state ... Both runs y2nb and 9gde reached executor.ts:2661 (which is INSIDE execute(), past the synchronous this.executing.add claim). The only viable explanation is that there is more than one TaskExecutor instance in the same Node process (engine restart race, multi-project hybrid runtime, or similar code path). Each instance has its own executing Set, so the per-instance guard doesn't help. Fix: module-level singleton executingTaskLock in active-session-registry.ts, shared across all TaskExecutor instances. execute() synchronously tryClaim()s the lock; if false, bails. Every existing this.executing.delete() site also calls executingTaskLock.release(). Per-instance this.executing kept because many other call sites use it (this.executing.has at handler gates, stuck-detector, resumeTaskForAgent, etc.). Test setup (resetExecutorMocks in executor-test-helpers.ts) clears the lock between tests so process-wide state doesn't leak (executor-pause and executor-prompt tests would otherwise show 'expected 2 createFnAgent calls but got 0' / 'expected not called but called 3 times' flakes). Tests: - executing-task-lock.test.ts: 2 cases. Key case creates TWO TaskExecutor instances and races them on the same task ID, asserts only ONE actually runs. Verified FAILS on prior code (8bef30655) and PASSES on fix. Verification: - Targeted suite (4 files, 170 tests): pass. - pnpm --filter @fusion/engine build: clean. - pnpm lint: clean. Fusion-Task-Id: FN-4811
1.2 KiB
@runfusion/fusion
| @runfusion/fusion |
|---|
| patch |
fix(FN-4811): use process-wide executingTaskLock to block parallel execute() across instances
After commit 82f80e72f added a per-instance this.executing.add() synchronous claim, production STILL produced two execute() invocations for the same task ID that both reached "Executor detected stale merge state" and both generated runIds within 1 second of each other (FN-4809: y2nb + 9gde at 02:48:17–18 UTC; FN-4814 / FN-4811 cascade). The only viable explanation is that there is more than one TaskExecutor instance in the process (engine restart race, multi-project hybrid runtime, etc.).
Adds a module-level singleton executingTaskLock in active-session-registry.ts shared across all TaskExecutor instances. TaskExecutor.execute() synchronously claims the lock immediately after the executorLog.log entry; if tryClaim() returns false (someone else owns the lock), the call bails. Every existing this.executing.delete() site also releases the lock. Per-instance this.executing is kept for back-compat with the many this.executing.has() checks throughout executor.ts.
Test setup in executor-test-helpers.ts clears the process-wide lock in resetExecutorMocks() so it doesn't leak across tests.