root/fusion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(engine): reconcile agents stuck in running state after missed heartbeat

Browse Source 
Recovery path now calls completeRun(terminated) so the canonical agent-state
transition runs, and reconcileOrphanedRunningAgents both catches stale-heartbeat
cases and runs every poll so pre-existing stuck rows self-heal post-upgrade.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
gsxdsm
2026-05-05 23:08:14 -07:00
parent e70b1c5d15
commit d2ca8aef17
2 changed files with 76 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
.changeset/heartbeat-reconcile-stuck-running.md Normal file
View File

@@ -0,0 +1,12 @@
---
"@fusion/engine": patch
---
Fix agents stuck in `state="running"` after a missed-heartbeat termination.
The unresponsive-agent recovery path disposed the session and called `pauseAgent`, but never explicitly ended the run via `completeRun` — relying on the in-flight execution to self-complete via its catch handler, which doesn't happen when the run is genuinely hung. The run record could still be terminated through other paths (safety-net or supersede-on-startRun), but those bypass the agent-state transition, leaving the agent permanently displayed as "running" with no active run.
Two fixes:
- `recoverUnresponsiveAgent` now calls `completeRun(..., status: "terminated")` so the canonical state transition runs alongside the existing `pauseAgent`/`resumeAgent` sequence.
- `reconcileOrphanedRunningAgents` is broadened to also catch agents with stale `lastHeartbeatAt` (> 3× timeout) that aren't in the in-memory tracked set, terminating their stale run record. It now runs every poll instead of only at monitor start, so any pre-existing stuck rows from older versions self-heal within one poll interval after upgrade.