root/fusion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Raise min heartbeat staleness floor 5m -> 10m

Browse Source 
Agents stop heartbeating during long legitimate work (e.g. a verification
step blocked on a multi-minute test command). The 5-minute floor could
misread a busy agent as dead and reclaim its in-progress task mid-run.
Raise MIN_HEARTBEAT_STALENESS_MS to 10 minutes and strengthen the floor
test (7-minute-silent fast-interval agent stays healthy — would have read
stale under the old 5-minute floor). Engine typecheck clean; heartbeat
suite 148/148 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
gsxdsm
2026-06-19 21:46:00 -07:00
parent 6c9e7554c1
commit e78853708a
3 changed files with 14 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.changeset/heartbeat-staleness-10min.md Normal file
View File

@@ -0,0 +1,5 @@
---
"@runfusion/fusion": patch
---
Raise the minimum agent heartbeat staleness floor from 5 to 10 minutes. Agents go silent during long-running but legitimate work (notably a verification step running a multi-minute test command, where the agent is blocked awaiting the command and cannot tick/heartbeat). The 5-minute floor could misread such a busy agent as dead and reclaim its in-progress task mid-run; 10 minutes gives long operations room before the liveness gate acts.

7
packages/engine/src/__tests__/heartbeat-executor.test.ts
View File

@@ -460,15 +460,18 @@ describe("executeHeartbeat", () => {
expect(section).toMatch(/\| Long Interval \| active \| FN-209 \| .* \| healthy \|/);
});
it("buildReportsHealthSection enforces a 5-minute minimum staleness floor", async () => {
it("buildReportsHealthSection enforces a 10-minute minimum staleness floor", async () => {
const now = Date.now();
const store = createStoreWithAgentForExec();
vi.mocked(store.getCachedAgent).mockImplementation((id: string) => ({
id,
runtimeConfig: { heartbeatIntervalMs: 1_000 },
}) as unknown as Agent);
// 7 minutes silent with a 1s interval: under the old 5-minute floor this
// would read as stale, but the 10-minute floor must still treat a busy
// long-running agent as healthy (e.g. mid multi-minute test command).
vi.mocked(store.getAgentsByReportsTo).mockResolvedValue([
{ id: "agent-fast", name: "Fast Poller", state: "active", taskId: "FN-210", lastHeartbeatAt: new Date(now - 2 * 60_000).toISOString(), updatedAt: new Date(now - 2 * 60_000).toISOString() } as Agent,
{ id: "agent-fast", name: "Fast Poller", state: "active", taskId: "FN-210", lastHeartbeatAt: new Date(now - 7 * 60_000).toISOString(), updatedAt: new Date(now - 7 * 60_000).toISOString() } as Agent,
]);
const monitor = new HeartbeatMonitor({ store, taskStore: mockTaskStore, rootDir: "/tmp" });

5
packages/engine/src/agent-heartbeat.ts
View File

@@ -204,8 +204,11 @@ const REPORTS_STALE_INTERVAL_MULTIPLIER = 1.5;
/**
* Minimum staleness threshold floor for very short heartbeat intervals.
* 10 minutes: long-running but legitimately-busy agents (e.g. a verification
* step running a multi-minute test command, during which the agent does not
* tick/heartbeat) must not be misread as dead and reclaimed mid-run.
*/
const MIN_HEARTBEAT_STALENESS_MS = 5 * 60_000;
const MIN_HEARTBEAT_STALENESS_MS = 10 * 60_000;
/** Format milliseconds into a human-readable duration string (e.g. "5m", "1h 20m", "2h"). */
export function formatDuration(ms: number): string {