Admit under-budget paused/heartbeat-model-unavailable agents to the shared
heartbeatErrorRecovery budget so timer, self-healing, and startup paths
retry without a manual Retry. Keep the pause reason when the budget is
exhausted so operators still see credential guidance.
Fixes agents silently going stale for hours even though the heartbeat repair audit process was running.
- HeartbeatTriggerScheduler now runs an independent watchdog (armTimerAuditWatchdog/checkTimerAuditLiveness) that tracks the audit loop's last-run timestamp and re-arms + immediately re-runs the 60s audit interval if it goes stale beyond a bounded multiple of the cadence, so a silently dropped audit driver self-heals instead of leaving active agents unrepaired for hours.
- Tracks consecutive non-advancing zombie-timer re-arms per agent (nonAdvancingRearmState) and escalates once the count crosses a threshold, recording consecutiveNonAdvancingRearms/nonAdvancingEscalated in agent.metadata.heartbeatTimerRepair and logging reason=heartbeat-rearm-nonadvancing-escalated instead of silently churning the same zombie-timer-rearmed repair forever.
- Clears non-advancing rearm state on unregister, non-eligible agents, paused settings, and stale-run-reap skip paths so tracking never leaks stale per-agent counters.
- Watchdog and its interval handle are armed in start() and cleared in stop() alongside the existing audit interval.
- Adds a changeset (patch) describing the fix, and updates docs/agents.md and docs/architecture.md to document the FN-7939 audit watchdog and non-advancing escalation behavior.
- Adds heartbeat-scheduler.test.ts coverage for watchdog re-arm/liveness and non-advancing escalation.
Files changed:
.changeset/fn-7939-heartbeat-audit-supervision.md | 7 +
docs/agents.md | 8 +-
docs/architecture.md | 1 +
.../src/__tests__/heartbeat-scheduler.test.ts | 209 +++++++++++++++++++++
packages/engine/src/agent-heartbeat.ts | 128 ++++++++++++-
5 files changed, 341 insertions(+), 12 deletions(-)
Fusion-Task-Id: FN-7939
Fusion-Task-Lineage: 9fa90240-4333-4588-b595-aef3811b1524
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Adds a best-effort, idempotent dashboard inbox notice announcing the upcoming embedded-Postgres storage migration, delivered once per project on the first engine start under the Fusion 0.59.x release line.
- New `deliverPostgresMigrationNoticeIfNeeded` in `@fusion/engine` (`postgres-migration-notice.ts`) builds and sends a `system` -> `user` inbox message via `MessageStore`, gated to version `0.59.x` by `isPostgresMigrationNoticeVersion`
- Idempotency via existing inbox message `metadata.kind = "postgres-migration-notice"` marker (no new settings key or table), so restarts never duplicate the notice
- Delivery is fully best-effort: any `MessageStore` failure is caught, logged as a warning, and never blocks or fails `ProjectEngine.start()`
- `ProjectEngine.start()` invokes the notice after runtime start, using an injected `cliPackageVersion` threaded from the CLI layer through `EngineManagerOptions` / `ProjectEngineOptions` so the engine never imports CLI/dashboard code directly
- `daemon.ts`, `dashboard.ts`, and `serve.ts` resolve the published `@runfusion/fusion` version via `getCliPackageVersion` / `isUnresolvedCliPackageVersion` and pass it into `ProjectEngineManager`
- Exported new symbols (`POSTGRES_MIGRATION_HELP_URL`, `POSTGRES_MIGRATION_NOTICE_KIND`, `deliverPostgresMigrationNoticeIfNeeded`, `isPostgresMigrationNoticeVersion`, related types) from `@fusion/engine`, and `isUnresolvedCliPackageVersion` from `@fusion/dashboard`
- New unit tests covering version matching and single-delivery/idempotency behavior
- Docs updated (`docs/agents.md`, `docs/dashboard-guide.md`) to describe the one-time notice and its dedup key
- Changeset added for `@runfusion/fusion` (minor, feature)
Files changed:
.changeset/fn-7879-postgres-migration-inbox-notice.md | 7 ++
docs/agents.md | 1 +
docs/dashboard-guide.md | 1 +
packages/cli/src/commands/daemon.ts | 6 +-
packages/cli/src/commands/dashboard.ts | 5 +
packages/cli/src/commands/serve.ts | 6 +-
packages/dashboard/src/index.ts | 2 +-
packages/engine/src/__tests__/postgres-migration-notice.test.ts | 140 +++++++++++++++++++++
packages/engine/src/index.ts | 9 ++
packages/engine/src/postgres-migration-notice.ts | 107 ++++++++++++++++
packages/engine/src/project-engine-manager.ts | 6 +
packages/engine/src/project-engine.ts | 12 ++
12 files changed, 299 insertions(+), 3 deletions(-)
Fusion-Task-Id: FN-7879
Fusion-Task-Lineage: 201877e5-6bdc-4168-a8ac-ae0e50ec8308
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Debug org agents error state recovery regression: durable heartbeat-managed
agents with a non-recoverable error (permanent/credential/model-access/
config, not stale-worktree/module-resolution) were previously left
indefinitely in bare `state:"error"` with no operator-visible reason,
and CLI agent inspection tools did not surface error/pause diagnostics.
- Timer path (`HeartbeatMonitor`) and run-entry recovery now classify
non-recoverable durable heartbeat errors and park the agent `paused`
with `pauseReason:"error-unrecoverable"` instead of restart-looping or
sitting in `error` forever.
- `SelfHealingManager` mirrors the same non-recoverable classification in
its recovery sweep, parking with the same reason/metadata and skipping
the exhausted/next-retry gates for that terminal bucket.
- New `agent:error-parked-unrecoverable` run-audit event type emitted by
both the heartbeat and self-healing paths (ids/counts/outcomes-only
metadata).
- `fn_agent_show` now prints `Last Error`, `Pause Reason`, and a compact
`Error Recovery` counter line; `fn_list_agents` prints the same
diagnostics only for agents currently in `error`/`paused`.
- Updated `AGENTS.md`, `docs/agents.md`, and `docs/architecture.md` to
document the new terminal-park behavior and CLI diagnostics surface.
- Added a changeset (`@runfusion/fusion` patch) describing the
operator-facing fix.
Files changed:
.changeset/fn-7859-org-agent-error-diagnostics.md | 7 ++
AGENTS.md | 2 +-
docs/agents.md | 3 +-
docs/architecture.md | 4 +-
packages/cli/src/__tests__/extension.test.ts | 68 ++++++++++++++++
packages/cli/src/extension.ts | 64 +++++++++++++++
.../src/__tests__/heartbeat-error-recovery.test.ts | 47 ++++++++++-
packages/engine/src/__tests__/self-healing.test.ts | 94 ++++++++++++++++++----
packages/engine/src/agent-heartbeat.ts | 71 +++++++++++++++-
packages/engine/src/run-audit.ts | 1 +
packages/engine/src/self-healing.ts | 46 +++++++++--
11 files changed, 375 insertions(+), 32 deletions(-)
Fusion-Task-Id: FN-7859
Fusion-Task-Lineage: 09b2035d-e8a0-438f-b1ab-1b0048b35c76
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Issue #2015: product-code executor tasks were repeatedly routed to a
liaison-only agent because every routing path gated only on the coarse
role field, and several binding primitives had no guard at all.
- Add runtimeConfig.assignmentPolicy ("auto" | "explicit-only" | "none");
"none" can never be bound to implementation tasks by ANY path — no
override bypasses it (the liaison guarantee)
- Route every binding surface through one shared evaluator
(evaluateImplementationTaskBind): claimTaskForAgent, the previously
unguarded checkoutTask/assignTask primitives, selectNextTaskForAgent
(including the in-progress re-selection loop), scheduler auto-assign
pool, heartbeat inbox/auto-claim, fn_delegate_task, CLI agent-id
validation, and dashboard assign/checkout/inbox routes
- Lock project isolation with a regression test: a foreign-project
agent id is rejected by every binding primitive
- Expose Assignment Policy in Agent Detail settings; document in
docs/agents.md; add changeset
Fusion-Task-Id: FN-7851
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unifies the two independent durable-agent error-recovery paths (heartbeat timer and self-healing sweep) so they share one retry budget, eligibility check, and audit surface instead of racing separate counters.
- Share the heartbeatErrorRecovery attempt budget between HeartbeatMonitor's timer-entry recovery and SelfHealingManager.recoverOrphanedAgents(), with self-healing's legacy durableErrorRecovery metadata folded into the same counter via readHeartbeatErrorRetryCount().
- Add isHeartbeatErrorRecoverable() as the single transient/non-operator-actionable eligibility check, used by both the heartbeat timer and self-healing paths (self-healing additionally allows stale-worktree module-resolution errors).
- resetHeartbeatErrorRecoveryMetadata() now strips the legacy durableErrorRecovery field so recovered agents don't retain stale sweep bookkeeping.
- Self-healing emits the shared agent:auto-recover-error-state / agent:error-retry-exhausted run-audit events with source:"self-healing", and parks the agent paused with pauseReason:"error-retry-exhausted" on budget exhaustion, matching the heartbeat-timer behavior.
- Update AGENTS.md, docs/architecture.md, and docs/agents.md to describe the consolidated recovery budget and audit surface.
- Add a patch changeset documenting the fix for @runfusion/fusion.
Files changed:
.changeset/fn-7844-error-recovery-coordination.md | 7 ++
AGENTS.md | 2 +-
docs/agents.md | 14 ++--
docs/architecture.md | 2 +-
packages/engine/src/__tests__/heartbeat-error-recovery.test.ts | 13 +++-
packages/engine/src/__tests__/self-healing.test.ts | 58 ++++++++++++++-
packages/engine/src/agent-heartbeat.ts | 35 ++++++---
packages/engine/src/self-healing.ts | 85 ++++++++++++++++++----
8 files changed, 180 insertions(+), 36 deletions(-)
Fusion-Task-Id: FN-7844
Fusion-Task-Lineage: b70dcba5-56b6-412c-8be2-ef827bee9964
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
- contain fn_artifact_register path payloads: realpath-canonicalized
containment before stat/read — relative paths require and must stay
inside baseDir, absolute paths allowed only under baseDir or the OS
temp dir (deliberate allowance for browser/screenshot tooling);
the process.cwd() fallback is gone, symlink escapes rejected
- bind task-scoped heartbeat artifact registration to the acquired
worktree (baseDir: sessionCwd rebind after acquisition); no-task
heartbeat prompt now says to pass absolute temp-dir paths
- enforce exactly-one payload source (content/uri/dataBase64/path);
content+uri combos are now rejected to match the documented contract
- add FNXC rationale comments at both visual-artifact instruction sites
in the planning prompts (sync contract with the executor prompt)
- media route: statSync -> await stat from node:fs/promises
- range tests ride the in-memory MockSocket harness (TestResponse gains
binary-safe bodyBuffer; real-TCP helper deleted) and assert the full
206 Content-Range/Content-Length contract for every range form
- add PdfViewer coverage (iframe src/title) in DocumentsView tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Video was registrable but effectively unusable, and HTML/PDF deliverables
had no first-class path from agents to the gallery.
- media route now serves HTTP byte ranges (Accept-Ranges, 206 +
Content-Range, 416 on unsatisfiable) so <video>/<audio> seeking works
and Safari plays media at all
- video attachments (mp4/webm/mov, 100MB cap vs 5MB for other types)
bridge into the artifact registry like images; multer transport ceiling
raised to 100MB with per-type caps enforced in the store
- fn_artifact_register path payloads are signature-validated for video
(ftyp box / EBML header) and PDF (%PDF- prefix), mirroring images
- HTML doc artifacts (mimeType text/html) render as live sandboxed
iframe previews by default in the doc viewer, with a Preview/Source
toggle and the same FileEditor edit mode
- executor/heartbeat/planning prompts and tool descriptions now cover
the full type matrix: images, videos, audio, HTML mockups, PDFs, and
markdown docs, each with the registration recipe
Verified live: range requests (200/206/416) via curl, an ffmpeg-generated
mp4 playing to completion in the gallery lightbox, and an interactive
HTML mockup rendering in the sandboxed preview.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Agents could never get screenshots/wireframes/mocks into the Artifacts view:
fn_artifact_register was gated on assignedAgentId (never set in default
ephemeral mode), the only image payload source was inline base64, and no
prompt ever told agents to register visual deliverables.
- always expose fn_artifact_register to executor sessions ("executor" author
fallback), resolve relative paths against the task worktree, and default
taskId to the executing task (heartbeat task lane too)
- add a `path` payload source: file read with 50MB cap, extension MIME
inference, PNG/JPEG/GIF/WebP signature + SVG sniff validation, persisted
through managed artifact storage
- executor/heartbeat/planning prompts + engine-tools reference now instruct
agents to register screenshots, wireframes, mockups, and recordings
- new ArtifactsGallery: Images/Docs/PDFs/Videos/Audio/Other category sections
and filter chips, visual tile grid + lightbox, embedded PDF viewer, audio
player rows, download rows; mobile-responsive down to the 768px breakpoint
- doc artifacts open a full viewer rendered as markdown by default with an
in-place edit mode using the shared CodeMirror FileEditor; persisted via new
GET/PATCH /api/artifacts/:id + TaskStore.updateArtifact and live-refreshed
through the new artifact:updated SSE event
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fixes plugin skills silently disappearing when the fn daemon is started outside the project that enabled the contributing plugin, by making skill resolution project-aware instead of scoped to the daemon's root PluginLoader.
- getPluginSkills now resolves per requesting rootDir against project_plugin_states rather than the daemon-root PluginLoader scope
- Plugins skipped as disabled are now logged at load time for visibility
- Wired the new project-aware resolution through dashboard.ts, serve.ts, and daemon.ts CLI commands
- Added regression coverage in plugin-loader.test.ts and skills-adapter.test.ts
- Documented the project-scoped behavior in docs/PLUGIN_AUTHORING.md and docs/agents.md
- Added a patch changeset for @runfusion/fusion
Files changed:
.changeset/fn-7778-plugin-skills-project-scope.md | 7 +++
docs/PLUGIN_AUTHORING.md | 2 +
docs/agents.md | 2 +-
packages/cli/src/commands/daemon.ts | 68 +++++++++++++++++++--
packages/cli/src/commands/dashboard.ts | 71 ++++++++++++++++++++--
packages/cli/src/commands/serve.ts | 68 +++++++++++++++++++--
packages/core/src/__tests__/plugin-loader.test.ts | 69 +++++++++++++++++++++
packages/core/src/plugin-loader.ts | 29 ++++++---
.../dashboard/src/__tests__/skills-adapter.test.ts | 29 +++++++++
packages/dashboard/src/skills-adapter.ts | 19 ++++--
10 files changed, 337 insertions(+), 27 deletions(-)
Fusion-Task-Id: FN-7778
Fusion-Task-Lineage: 5d9a8ff2-ed0e-4859-bf9c-a16f715b081d
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Adds opt-in cross-process change detection to AgentStore so the engine reacts to CLI-driven agent stop/start mutations promptly instead of waiting for the periodic audit sweep.
- AgentStore gains fs.watch-based (with poll fallback) cross-process notification, modeled on TaskStore's existing mechanism
- Re-emits existing agent:updated/agent:stateChanged events in the engine process when another process (the fn CLI) mutates an agent row
- HeartbeatTriggerScheduler listeners now fire immediately instead of waiting up to 60s for the auditTimerRegistrations sweep; the sweep remains as durable backstop
- in-process-runtime.ts wires up the new notification bus
- Adds unit tests for agent-store cross-process notifications and heartbeat-scheduler reaction behavior
- Updates docs/agents.md and docs/architecture.md
- Adds changeset (patch) for @runfusion/fusion
Files changed:
.changeset/fn-7723-cross-process-agent-notify.md | 7 +
docs/agents.md | 1 +
docs/architecture.md | 1 +
packages/core/src/__tests__/agent-store.test.ts | 177 +++++++++++++++++
packages/core/src/agent-store.ts | 210 ++++++++++++++++++++-
.../src/__tests__/heartbeat-scheduler.test.ts | 162 ++++++++++++++++
packages/engine/src/runtimes/in-process-runtime.ts | 30 +++
7 files changed, 587 insertions(+), 1 deletion(-)
Fusion-Task-Id: FN-7723
Fusion-Task-Lineage: d3a7fa05-b40d-4388-8e98-140f9d8861c9
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Ensures stopping and restarting an agent durably clears its heartbeat timer instead of relying on the later FN-7645 watchdog repair.
- HeartbeatTriggerScheduler.auditTimerRegistrations now unregisters lingering timers for non-eligible (stopped/paused/disabled) agents
- syncTimerForAgent force-re-arms a stale present timer on a start transition so no orphaned timer entry lingers
- Added 308 lines of new heartbeat-scheduler regression tests covering the stop/start zombie-timer scenarios
- Added changeset (patch) documenting the fix
- Updated docs/agents.md and docs/architecture.md to describe the new invariant
Files changed:
.changeset/fn-7718-zombie-timer-invalidate.md | 7 +
docs/agents.md | 2 +
docs/architecture.md | 1 +
.../src/__tests__/heartbeat-scheduler.test.ts | 308 +++++++++++++++++++++
packages/engine/src/agent-heartbeat.ts | 49 +++-
5 files changed, 364 insertions(+), 3 deletions(-)
Fusion-Task-Id: FN-7718
Fusion-Task-Lineage: fc834ccd-495e-4294-805d-325b4cb536a2
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Agents were composing plans (e.g. reboot/wait-and-retry loops) that assumed they could keep acting even after the Fusion platform itself shut down, since prompts never told them they run inside Fusion. This adds a shared, docs-grounded self-awareness preamble prepended to chat, heartbeat, and executor base prompts so agents know their own runtime constraints.
- Added FUSION_RUNTIME_SELF_AWARENESS shared preamble in packages/core/src/agent-prompts.ts, exported via packages/core/src/index.ts
- Prepended the preamble to the chat system prompt (packages/dashboard/src/chat.ts)
- Prepended the preamble to the heartbeat session prompt (packages/engine/src/agent-heartbeat.ts)
- Prepended the preamble to the executor base prompt (packages/engine/src/executor.ts)
- Updated docs/agents.md and CONCEPTS.md to document the new self-awareness/capability-grounding behavior
- Added regression tests across core, dashboard, and engine covering the new prompt content
- Added changeset for @runfusion/fusion (minor, fix category)
Files changed:
.changeset/fn-7675-agent-runtime-self-awareness.md | 7 ++++
CONCEPTS.md | 4 +-
docs/agents.md | 17 ++++++++
packages/core/src/__tests__/agent-prompts.test.ts | 41 ++++++++++++++++++++
packages/core/src/agent-prompts.ts | 32 ++++++++++++++-
packages/core/src/index.ts | 1 +
packages/dashboard/src/__tests__/chat-system-prompt.test.ts | 17 ++++++++
packages/dashboard/src/chat.ts | 6 ++-
packages/engine/src/__tests__/executor-prompt.test.ts | 45 ++++++++++++++++++++++
packages/engine/src/__tests__/heartbeat-session-prompt.test.ts | 35 +++++++++++++++++
packages/engine/src/agent-heartbeat.ts | 10 +++--
packages/engine/src/executor.ts | 7 +++-
12 files changed, 213 insertions(+), 9 deletions(-)
Fusion-Task-Id: FN-7675
Fusion-Task-Lineage: 126d04a6-2c68-4347-9789-591b274277bf
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Root-causes 4 correlated CTO-report agent failures where durable non-ephemeral agents got stuck in `error` state indefinitely because the heartbeat scheduler stops ticking error-state agents entirely, and self-healing's recovery sweep previously only considered them when their manager row was missing.
- SelfHealingManager: scope the `managerMissing` gate to the "running" orphan-detection path only, so "error"-state durable agents with a present/active manager now fall through to the existing transient/operator-actionable/active-execution/cooldown/retry-budget recovery guards instead of being skipped outright
- Add FNXC:AgentHeartbeat comment documenting the FN-7672 incident and rationale for the scoping change
- Extend self-healing.test.ts with coverage for manager-present durable agents in error state
- Add changeset (patch) describing the fix for release notes
- Update docs/agents.md accordingly
Files changed:
.changeset/fn-7672-durable-agent-recovery.md | 7 ++
docs/agents.md | 2 +
packages/engine/src/__tests__/self-healing.test.ts | 129 ++++++++++++++++++++-
packages/engine/src/self-healing.ts | 24 +++-
4 files changed, 160 insertions(+), 2 deletions(-)
Fusion-Task-Id: FN-7672
Fusion-Task-Lineage: 6676dc9e-66e7-4f70-804a-cccf77e8d337
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Executors could previously treat a pending approval as a normal turn end and go hunt for ungated workarounds instead of stopping. This change makes wait-for-approval a hard suspend point.
- wait-for-approval now suspends the in-flight executor session via awaitAbortInFlightTaskWork
- Dedupe identical pending approvals so repeated waits don't pile up
- Executor prompts now carve out awaiting-approval as a legitimate turn end (agent-prompts.ts)
- Extend provisioning-gate and agent-action-gate coverage for the new suspend/carveout behavior
- Add changeset (patch) documenting the fix for release notes
- Update docs/agents.md and docs/architecture.md to describe the new blocking behavior
Files changed:
.changeset/fn-7608-awaiting-approval-blocking.md | 7 ++
docs/agents.md | 1 +
docs/architecture.md | 1 +
packages/core/src/agent-prompts.ts | 5 +
.../engine/src/__tests__/agent-action-gate.test.ts | 82 +++++++++++++
.../executor-approval-gate-suspend.test.ts | 128 +++++++++++++++++++++
.../executor-approval-prompt-carveout.test.ts | 61 ++++++++++
packages/engine/src/agent-heartbeat.ts | 13 +++
packages/engine/src/executor.ts | 28 +++++
packages/engine/src/pi.ts | 22 +++-
.../sandbox/__tests__/provisioning-gate.test.ts | 29 +++++
packages/engine/src/sandbox/provisioning-gate.ts | 11 ++
12 files changed, 384 insertions(+), 4 deletions(-)
Fusion-Task-Id: FN-7608
Fusion-Task-Lineage: 9e42d8ee-bda7-4ef1-b159-46c2100bbc48
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Classify fn_task_create as a task-agent mutation so permanent-agent policies can require approval or block task creation.
- Move fn_task_create out of the permanent-agent read-only tool set and into the shared task mutation classification.
- Add parity tests covering permanent-agent and action-gate decisions for fn_task_create.
- Preserve delegate and GitHub import tools as permanent-agent read-only coordination tools while documenting the distinction.
- Add a patch changeset for the published Fusion package.
Files changed:
.../fn-7132-task-create-mutation-classification.md | 7 +++
docs/agents.md | 6 +--
.../src/__tests__/gating-classifications.test.ts | 58 +++++++++++++++++++++-
.../src/__tests__/permanent-agent-gating.test.ts | 3 +-
packages/engine/src/gating-classifications.ts | 10 ++--
5 files changed, 75 insertions(+), 9 deletions(-)
Fusion-Task-Id: FN-7132
Fusion-Task-Lineage: 6e32ed9d-f0fa-46f2-a749-c5dbe863fb1f
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
