fb0863f660c19cbbe8796b0b53a69072d424a85b
451 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
0e69ed9a5b |
fix: stop stale-active-branch rescue spam for done squash leftovers (#3311)
## Summary - Local `fusion/*` branches for **done** tasks kept unique tip SHAs after squash/AI merge, so self-healing logged `stale-active-branch-rescue-needed` on every maintenance sweep without ever deleting them. - **Completion fan-out** now force-deletes the task branch even when unique commits remain (squash-safe). - **`reclaim-stale-active-branches`** force-deletes complete-lane leftovers (`reason=complete-column-unique-commits-force`) and no longer emits rescue-needed for those lanes; non-complete columns still warn and preserve unmerged work. Archived lanes still skip reclaim entirely. Also cleaned up 116 leftover local fusion branches on this machine (done/orphan only; 9 active kept). ## Test plan - [x] `pnpm --filter @fusion/engine exec vitest run src/__tests__/self-healing-completion-fanout.test.ts src/__tests__/self-healing.test.ts -t "SelfHealingManager reclaimStaleActiveBranches|self-healing completion fan-out|force-deletes"` - [ ] After merge/restart engine: confirm logs no longer spam rescue-needed for done tasks - [ ] Confirm active todo/in-progress fusion branches still get rescue-needed when unique and no worktree <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved cleanup of stale task branches after completion, including branches with unique commits remaining after squash or AI-assisted merges. * Completed tasks now reliably remove residual branches and worktree metadata. * Non-completed tasks continue to preserve recoverable branches and display rescue warnings. * Added clearer recovery logging and audit records for forced branch cleanup. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
6cc687d3a3 |
FN-8728: retry incomplete planner runs before review
Retry planner attempts until a clean, changed artifact is ready for Plan Review. - Track each planning attempt's prompt baseline and fallback provenance. - Require a runtime-owned fallback-dispatch boundary before review handoff. - Back off and surface actionable errors for empty, unchanged, or fallback-engaged attempts. Files changed: .changeset/fn-8728-planner-retry-before-review.md | 7 + docs/architecture.md | 2 +- packages/engine/src/__tests__/triage.test.ts | 492 ++++++++++++++++++++++ packages/engine/src/agent-runtime.ts | 8 + packages/engine/src/agent-session-helpers.ts | 3 + packages/engine/src/pi.ts | 14 +- packages/engine/src/runtime-resolution.ts | 14 +- packages/engine/src/triage.ts | 135 +++++- 8 files changed, 656 insertions(+), 19 deletions(-) Fusion-Task-Id: FN-8728 Fusion-Task-Lineage: e7581d29-5bd6-43dc-aa0b-6594b9dd9c0d Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
01d65805c3 |
FN-8693: refresh reused worktree bases before execution
Refresh reused execution worktrees against the current integration baseline. - Rebase or reset clean reused worktrees before coding sessions while preserving task commits. - Persist and audit refreshed base SHAs, and block unsafe refresh states before execution. - Cover executor, graph, and heartbeat refresh paths with regression tests. Files changed: .changeset/fn-8693-stale-worktree-base.md | 7 + docs/architecture.md | 1 + .../src/__tests__/agent-heartbeat-worktree.test.ts | 28 ++++ .../__tests__/ce-workflow-step-executor.test.ts | 44 ++++++ .../src/__tests__/worktree-base-refresh.test.ts | 90 ++++++++++++ packages/engine/src/agent-heartbeat.ts | 35 ++++- packages/engine/src/executor.ts | 67 ++++++++- packages/engine/src/merger.ts | 5 + packages/engine/src/run-audit.ts | 11 ++ packages/engine/src/workflow-graph-executor.ts | 32 ++++- packages/engine/src/worktree-acquisition.ts | 28 +++- packages/engine/src/worktree-base-refresh.ts | 158 +++++++++++++++++++++ 12 files changed, 498 insertions(+), 8 deletions(-) Fusion-Task-Id: FN-8693 Fusion-Task-Lineage: e39a441f-39b5-4723-b503-753e921018f3 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
e8ca86d5ee |
FN-8705: prioritize review and execution slot admission
Prioritize lifecycle-critical work whenever project capacity becomes available. - Rank admission candidates as review/merge, execution, then planning. - Coordinate scheduler handoffs with project and host capacity reservations. - Cover lane priority and document the updated operator behavior. Files changed: .changeset/fn-8705-slot-priority.md | 7 ++ docs/architecture.md | 2 +- docs/dashboard-guide.md | 2 +- packages/engine/src/__tests__/concurrency.test.ts | 95 +++++++++++----------- .../engine/src/__tests__/project-engine.test.ts | 4 +- .../starved-refinement-x-triage-poll.test.ts | 11 ++- .../__tests__/triage-refinement-routing.test.ts | 16 ++-- .../workflow-continuation-capacity.test.ts | 3 +- packages/engine/src/concurrency.ts | 37 +++++++-- packages/engine/src/project-engine.ts | 10 ++- packages/engine/src/runtimes/in-process-runtime.ts | 3 +- packages/engine/src/scheduler.ts | 40 ++++----- packages/engine/src/triage.ts | 7 +- 13 files changed, 133 insertions(+), 104 deletions(-) Fusion-Task-Id: FN-8705 Fusion-Task-Lineage: ef66360b-e504-4e3f-b25e-b032a719d8c0 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
ac8ce148d5 |
FN-8691: suppress repeated task wedge notifications
Prevent resolved-and-rewedged tasks from flooding operator notification channels. - Persist per-reason notification timestamps with a six-hour cooldown. - Apply cooldown claims consistently in durable storage and in-memory fallback paths. - Cover reason transitions, legacy state, invalid timestamps, and cooldown expiry. Files changed: .changeset/fn-8691-wedge-notification-cooldown.md | 7 ++ docs/architecture.md | 4 +- .../postgres/store-wedge-resolution.pg.test.ts | 67 +++++++++++- packages/core/src/index.ts | 1 + packages/core/src/store.ts | 21 +++- packages/core/src/task-store/task-mutation-ops.ts | 6 ++ packages/core/src/types.ts | 2 + packages/core/src/types/task-core.ts | 15 +++ .../__tests__/task-wedge-notification.test.ts | 117 ++++++++++++++++++++- .../src/notification/notification-service.ts | 30 +++++- 10 files changed, 261 insertions(+), 9 deletions(-) Fusion-Task-Id: FN-8691 Fusion-Task-Lineage: b7460508-f9b0-488b-828c-a51e9477304d Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
04c2bb4707 |
FN-8654: rotate credential instances after provider limits
Retry provider-limit failures with eligible credential instances before falling back to existing pauses and backoff. - Add a runtime-shared credential rotator with cooldown, exhaustion, and audit handling. - Wire credential rotation into executor and heartbeat retry lanes while preserving user pause controls. - Document the behavior and cover rotation, recovery, and retry paths. Files changed: .changeset/fn-8654-credential-instance-rotation.md | 7 + AGENTS.md | 1 + docs/architecture.md | 2 +- docs/settings-reference.md | 4 + .../__tests__/credential-instance-rotation.test.ts | 88 +++++++++++ .../__tests__/credential-rotation-lanes.test.ts | 20 +++ .../__tests__/credential-rotation-recovery.test.ts | 19 +++ .../__tests__/credential-rotation-wiring.test.ts | 15 ++ .../__tests__/rate-limit-retry-rotation.test.ts | 50 ++++++ .../src/__tests__/usage-limit-detector.test.ts | 14 ++ packages/engine/src/agent-heartbeat.ts | 102 +++++++++++- .../engine/src/credential-instance-rotation.ts | 175 +++++++++++++++++++++ packages/engine/src/executor.ts | 141 +++++++++++++++-- packages/engine/src/index.ts | 7 + packages/engine/src/project-engine.ts | 5 + packages/engine/src/rate-limit-retry.ts | 32 +++- packages/engine/src/runtimes/in-process-runtime.ts | 29 +++- packages/engine/src/usage-limit-detector.ts | 18 ++- 18 files changed, 699 insertions(+), 30 deletions(-) Fusion-Task-Id: FN-8654 Fusion-Task-Lineage: 44d63441-270c-4949-8c34-47ec4c9992e4 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
006cc40454 |
FN-8685: add durable cross-process task deletion consumers
Deliver durable, replay-safe cross-process task deletion observation. - Add PostgreSQL lifecycle consumer cursors, leases, acknowledgements, retention, and recovery. - Start named consumers in dashboard, serve, and engine runtime paths. - Preserve delete integration metadata while suppressing replayed GitHub and GitLab side effects. - Cover outbox identity, observed delivery, fencing, and reconciliation behavior. Files changed: ...fn-8685-cross-process-task-deleted-observers.md | 7 + .../fn-8685-task-deleted-outbox-consumers.md | 7 + docs/architecture.md | 8 +- ...tgres-cross-process-task-deleted-observation.md | 8 +- docs/storage.md | 10 +- packages/cli/src/commands/dashboard.ts | 9 +- packages/cli/src/commands/serve.ts | 9 +- packages/cli/src/project-context.ts | 9 +- .../task-deleted-outbox-consumer.pg.test.ts | 157 ++++++++ ...-deleted-observed-dispatch-side-effects.test.ts | 36 ++ .../task-lifecycle-consumer-identity.test.ts | 22 ++ packages/core/src/index.ts | 11 + .../0041_fn_8685_task_lifecycle_consumers.sql | 88 +++++ packages/core/src/postgres/schema-applier.ts | 16 +- packages/core/src/postgres/schema/project.ts | 45 +++ packages/core/src/postgres/startup-factory.ts | 4 + packages/core/src/store.ts | 54 ++- .../__tests__/lifecycle-outbox-writer.test.ts | 4 +- .../core/src/task-store/archive-lifecycle-2.ts | 1 + packages/core/src/task-store/lifecycle-ops.ts | 13 +- packages/core/src/task-store/lifecycle-outbox.ts | 2 + packages/core/src/task-store/project-store-ops.ts | 4 +- .../src/task-store/task-deleted-outbox-consumer.ts | 333 +++++++++++++++++ .../task-store/task-lifecycle-consumer-identity.ts | 32 ++ .../task-store/task-lifecycle-consumer-registry.ts | 396 +++++++++++++++++++++ .../task-store/task-lifecycle-event-retention.ts | 104 ++++++ packages/core/src/task-store/task-mutation-ops.ts | 1 + packages/dashboard/src/github-tracking-state.ts | 12 +- packages/dashboard/src/gitlab-delete-close.ts | 3 + packages/dashboard/src/gitlab-split-close.ts | 7 +- packages/dashboard/src/project-store-resolver.ts | 9 +- packages/engine/src/project-manager.ts | 4 +- packages/engine/src/project-runtime.ts | 2 +- packages/engine/src/runtimes/in-process-runtime.ts | 17 +- packages/engine/src/self-healing.ts | 27 ++ 35 files changed, 1439 insertions(+), 32 deletions(-) Fusion-Task-Id: FN-8685 Fusion-Task-Lineage: 63eca9ac-d2af-44b0-ba79-388a950148d3 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
8c9346ee94 |
FN-8684: persist task deletion events transactionally
Persist task-deletion lifecycle events through a transactional PostgreSQL outbox. - Add the task lifecycle outbox schema, migration, and atomic writer. - Route deletion notice persistence through the transaction and preserve non-blocking cleanup. - Cover outbox behavior, schema installation, and caller attribution with tests. Files changed: .changeset/fn-8684-task-deleted-outbox-writer.md | 7 + docs/architecture.md | 6 + docs/storage.md | 6 + .../src/__tests__/postgres/schema-applier.test.ts | 82 ++++++++- .../task-delete-caller-attribution.test.ts | 12 +- .../task-delete-nonblocking-cleanup.test.ts | 13 +- .../core/src/__tests__/task-delete-notice.test.ts | 7 +- .../0040_fn_8684_task_lifecycle_outbox.sql | 44 +++++ packages/core/src/postgres/schema-applier.ts | 15 +- packages/core/src/postgres/schema/project.ts | 27 +++ .../__tests__/lifecycle-outbox-writer.test.ts | 201 +++++++++++++++++++++ .../core/src/task-store/archive-lifecycle-2.ts | 93 ++++++++-- packages/core/src/task-store/async-persistence.ts | 26 ++- packages/core/src/task-store/lifecycle-outbox.ts | 46 +++++ 14 files changed, 557 insertions(+), 28 deletions(-) Fusion-Task-Id: FN-8684 Fusion-Task-Lineage: 1869d221-2d8a-48fb-b245-9cd1af56bda0 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
4009eb34cb |
FN-8683: remove unreachable SQLite task polling replica
Document PostgreSQL task-deletion observation and remove the obsolete SQLite polling path. - Remove polling state, replica emissions, and activity-log suppression from TaskStore. - Retain backend-aware cache warming while documenting the transactional-outbox follow-up. - Add tombstone and soft-delete abort coverage across core and engine lanes. Files changed: docs/architecture.md | 3 +- ...tgres-cross-process-task-deleted-observation.md | 128 ++++++++++++++++ docs/storage.md | 3 +- .../task-delete-nonblocking-cleanup.test.ts | 54 +++++++ .../task-deleted-polling-replica-tombstone.test.ts | 57 +++++++ .../task-updated-lanes-emit-surfaces.test.ts | 26 ---- packages/core/src/store.ts | 13 +- packages/core/src/task-store/lifecycle-ops.ts | 168 ++------------------- packages/core/src/task-store/task-artifacts-ops.ts | 4 - .../__tests__/executor-soft-delete-abort.test.ts | 15 ++ .../src/__tests__/triage-soft-delete-abort.test.ts | 14 ++ 11 files changed, 284 insertions(+), 201 deletions(-) Fusion-Task-Id: FN-8683 Fusion-Task-Lineage: a052db0c-b6fc-4b05-b6b2-f8217b56ded0 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
8a6949dd24 |
FN-8661: resolve selected credential instances for sessions
Resolve requested provider credential instances before creating agent sessions. - Thread lane credential instance selections through planning, validation, execution, review, and merge sessions. - Resolve selected instances into runtime credential stores while retaining provider-default fallback behavior. - Preserve selected credentials for mission validation, executor retries, and spawned child agents. Files changed: .../fn-8661-credential-instance-resolution.md | 7 ++ AGENTS.md | 1 + docs/architecture.md | 2 +- docs/secrets.md | 2 + docs/settings-reference.md | 1 + .../dashboard/src/__tests__/routes-auth.test.ts | 80 +++++++++++++++++++ .../dashboard/src/routes/register-model-routes.ts | 78 +++++++++++++++++++ .../src/__tests__/agent-session-helpers.test.ts | 16 ++++ .../credential-instance-resolution.test.ts | 49 ++++++++++++ packages/engine/src/agent-heartbeat.ts | 1 + packages/engine/src/agent-runtime.ts | 9 ++- packages/engine/src/agent-session-helpers.ts | 67 +++++++++++----- packages/engine/src/auth-storage.ts | 90 ++++++++++++++++++---- packages/engine/src/executor.ts | 29 ++++++- packages/engine/src/merger-ai.ts | 2 + packages/engine/src/merger.ts | 5 ++ packages/engine/src/mission-execution-loop.ts | 4 +- packages/engine/src/pi.ts | 7 +- packages/engine/src/pr-response-run-ops.ts | 1 + packages/engine/src/reviewer.ts | 7 ++ packages/engine/src/triage.ts | 2 + 21 files changed, 420 insertions(+), 40 deletions(-) Fusion-Task-Id: FN-8661 Fusion-Task-Lineage: 1e34a3ce-0857-4619-9746-ce0dc12dc2ba Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
0e3d2a2265 |
refactor: delete meta-task auto-archive and automated recovery follow-ups (#2461)
Deletes two pieces of automated "meta" machinery that filed and garbage-collected cards restating state already on the task that failed. Net **-1015 lines**. ## Why **Automated recovery follow-ups.** `createAutomatedFollowup` and its dedup engine (289 lines of signature matching, 1h recurrence rate-limiting, 24h supersedes windows) existed to file recovery cards for verification-cap and merge-conflict give-ups. In both cases the parent is *already* parked `failed` with a descriptive `error` and a log entry carrying the failing command, branch, and output — the card was a second copy of that. **Meta-task auto-archive.** The sweeps that garbage-collected those cards were worse than redundant: the regex classifier matched ordinary feature work, and its positional fallback bound cards to unrelated tasks, so **live work could be archived**. They are removed together, because the auto-archive sweeps only existed to clean up after the follow-up engine. ## What changed ### Deleted - `packages/engine/src/verification-followup-dedup.ts` in full — `createAutomatedFollowup`, `decideAutomatedFollowup`, `AutomatedFollowupKind`, `computeVerificationFailureSignature`, `extractFailingTestFiles`. - `findActiveRecoveryFollowUp` — dead code, defined and never called (`tsc` independently flagged it `6133 declared but its value is never read`). - The meta-task auto-archive sweeps `autoArchiveResolvedMetaTasks` / `autoArchiveStalledMetaTasks` and helpers `classifyMetaTask` / `resolveMetaTargetTaskId` / `computeMetaChainDepth` / `archiveMetaTask` / `evaluateMetaAutoArchiveGuards`, plus settings `metaTaskStallAutoCloseMs` and `metaTaskActiveExecutionGraceMs`. - Run-audit types `task:auto-archived-meta-resolved`, `task:auto-archived-meta-stalled`, `task:auto-archive-meta-resolved-skipped`, `task:auto-archive-meta-stalled-skipped`, `verification:followup-created`, `verification:followup-deduped`. The two signature helpers were **deleted rather than relocated** — once the three call sites went they were provably unreachable: `buildVerificationFailureSignature` had exactly one caller, and it was the only caller of `extractFailingTestFiles`. ### Call sites 1 and 2 — park kept, card dropped Verification-cap and merge-conflict give-ups keep their park, audit event, operator comment, and log entry. Site 1's `error` string was reworded off `"See follow-up task for investigation."` (no follow-up will exist) to carry the guidance itself. `autoResolveDisabled` was **kept** — it still drives the outer park guard and the `reason` string; only the inner branch that guarded card creation is gone. ### Call site 3 — autostash orphan, replaced not deleted This one is a genuine data-loss guard, so it keeps a durable trail. A `live`-classified orphan is a merger stash holding **real uncommitted work**, and unlike sites 1–2 there is no parked parent — the parent may already be `done` and merged, so nothing else on the board would ever mention the stash. The card is replaced by a `logEntry` **and** an `addTaskComment` on the parent, preserving every fact the old description carried: the sha, `record.label` (the handle `git stash` recovery needs), `record.detectedByTaskId`, and `sourcePhase`. New truthful run-audit event `task:autostash-orphan-live-detected` replaces the borrowed `verification:followup-*` name, with ids/outcomes-only metadata per AGENTS.md. ### Kept unchanged: the two real product features Eval follow-ups (`eval-followups.ts`) and PR-comment follow-ups (`pr-comment-handler.ts`) only borrowed the shared engine for its dedup pass. Both keep their exact behavior, column, priority, `sourceType`, and log lines, with dedup inlined as a `listTasks` scan on `suggestionId` / `prNumber` respectively. Both fail open (create) if the listing throws, matching the old engine. ## Test changes — read this one Two tests asserted the *deleted* engine's rate-limited `"[verification recurrence]"` logEntry. Those assertions were removed, **not loosened**: both tests still assert no duplicate card is created, and the eval test still asserts the existing id is reported back. No coverage of surviving behavior was weakened. The three `meta-*` test files were deleted along with the sweeps they covered. ## Verification ``` $ pnpm test:gate Test Files 2 passed (2) Tests 10 passed (10) # core Test Files 16 passed (16) Tests 299 passed (299) # engine-core Test Files 1 passed (1) Tests 70 passed (70) # ci-shape GATE_EXIT=0 $ pnpm --filter @fusion/engine --filter @fusion/core exec tsc --noEmit -p tsconfig.json TSC_EXIT=0 (no output) ``` Plus a file-scoped run over the touched surfaces (`eval-followups`, `pr-comment-handler`, `merger-autostash-orphan-surface`, `merger-autostash-cleanup`, `run-audit`, `run-audit-secret-taxonomy`, `project-engine`, `project-engine-manager`): **213/213 passed**. A repo-wide grep confirms no surviving references to any deleted symbol, module, or audit event. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Failed tasks now retain recovery and verification details directly on the original task instead of generating separate follow-up cards. * Live autostash issues now preserve stash information in task comments and activity logs. * Existing evaluation and pull-request follow-ups continue to be reused when appropriate. * **Changes** * Removed automatic archival of meta-tasks. * Removed obsolete meta-task timing settings. * **Documentation** * Updated architecture and settings documentation to reflect these workflow changes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
fd073e287f |
FN-8592: self-heal stranded hold continuations
Restore graph-owned plan-review continuations for eligible hold-column cards stranded after planning cancellation. - Detect real-spec hold cards with no active workflow continuation and re-seed Plan Review safely. - Serialize workflow continuation seeding, review-result writes, and lease claims to prevent duplicate recovery. - Add recovery diagnostics, release warnings, regression coverage, and a patch changeset. Files changed: .changeset/fn-8592-stranded-hold-continuation.md | 7 + AGENTS.md | 1 + docs/architecture.md | 4 + .../workflow-task-serialization-protocol.test.ts | 119 +++++++++++++ .../workflow-work-items-conditional-seed.test.ts | 191 +++++++++++++++++++++ packages/core/src/store.ts | 5 +- .../src/task-store/async-workflow-workitems.ts | 123 +++++++++---- packages/core/src/task-store/project-store-ops.ts | 14 ++ .../src/task-store/workflow-task-create-ops.ts | 16 +- .../src/task-store/workflow-workitems-ops-2.ts | 91 ++++++---- .../src/__tests__/pre-release-plan-review.test.ts | 17 ++ ...self-healing-stranded-hold-continuation.test.ts | 171 ++++++++++++++++++ packages/engine/src/hold-release.ts | 57 +++++- packages/engine/src/plan-review-continuation.ts | 94 ++++++++++ packages/engine/src/runtimes/in-process-runtime.ts | 30 +--- packages/engine/src/self-healing.ts | 100 ++++++++++- 16 files changed, 945 insertions(+), 95 deletions(-) Fusion-Task-Id: FN-8592 Fusion-Task-Lineage: fe7ffd34-96e4-4418-a879-7418e6293d30 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
0056d75314 |
FN-8569: surface unrecoverable report health
Classify parked direct reports as operator-actionable even when their stored state appears live. - Add a reusable Reports Health classifier that prioritizes pause markers. - Clear stale pause markers during live-state resumes without removing diagnostic errors. - Cover desynchronized report states and document the health invariant. Files changed: .../fn-8569-reports-health-error-unrecoverable.md | 7 +++ docs/architecture.md | 1 + .../agent-store-pause-marker-clear.test.ts | 65 +++++++++++++++++++ packages/core/src/agent-store.ts | 12 ++++ .../src/__tests__/heartbeat-executor.test.ts | 27 ++++++-- .../engine/src/__tests__/reports-health.test.ts | 73 ++++++++++++++++++++++ packages/engine/src/agent-heartbeat.ts | 36 ++++++----- packages/engine/src/index.ts | 6 ++ packages/engine/src/reports-health.ts | 70 +++++++++++++++++++++ 9 files changed, 278 insertions(+), 19 deletions(-) Fusion-Task-Id: FN-8569 Fusion-Task-Lineage: 37c798a0-1f2b-4221-a0c6-ccbff8d72696 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
96a9da7979 |
FN-8505: notify operators of terminal task wedges
Deliver durable, actionable notifications when terminal task recovery wedges. - Persist and deduplicate terminal wedge notification episodes across task updates and service restarts. - Classify terminal failure and self-healing escalation states, then deliver actionable ntfy and mailbox alerts. - Align PostgreSQL baseline and upgrade migration registration for the durable wedge field. Files changed: .changeset/fn-8505-task-wedge-notifications.md | 7 + docs/agents.md | 4 + docs/architecture.md | 4 + .../core/src/postgres/migrations/0000_initial.sql | 2 + .../migrations/0033_fn-8505_wedge_notification.sql | 5 + packages/core/src/postgres/schema-applier.ts | 29 +++- packages/core/src/postgres/schema/project.ts | 1 + packages/core/src/store.ts | 22 ++- packages/core/src/task-store/persistence.ts | 2 + packages/core/src/task-store/serialization.ts | 1 + packages/core/src/task-store/task-row-mappers.ts | 2 +- packages/core/src/task-store/task-update.ts | 5 + packages/core/src/types.ts | 14 ++ packages/core/src/types/workflow-steps.ts | 2 + .../src/__tests__/notification-service.test.ts | 20 +++ packages/engine/src/__tests__/notifier.test.ts | 26 +++- packages/engine/src/__tests__/self-healing.test.ts | 9 +- .../__tests__/notification-service.test.ts | 34 ++++- .../__tests__/task-wedge-notification.test.ts | 134 +++++++++++++++++ .../src/notification/notification-service.ts | 108 +++++++++++++- packages/engine/src/notification/ntfy-provider.ts | 11 ++ .../src/notification/task-wedge-notification.ts | 160 +++++++++++++++++++++ packages/engine/src/notifier.ts | 3 + packages/engine/src/self-healing.ts | 17 +++ 24 files changed, 610 insertions(+), 12 deletions(-) Fusion-Task-Id: FN-8505 Fusion-Task-Lineage: eee85220-18ba-475d-9d01-dc96e2b923e6 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
56efd7488e |
fix(engine): stop false-positive stuck loop kills on iterative work (#2404)
## Summary - Fix a false-positive in `StuckTaskDetector` where legitimate long single-step work (E2E debugging, iterative fix/test cycles) was classified as a loop and kill/requeued. - Root cause: loop meant “no step status transition for `taskStuckTimeoutMs` + high activity volume,” conflating **step progress** with **actual activity**. Agents can stay productively busy on one step for 10+ minutes with zero repetition. - Loop now requires thrash evidence on top of volume + no step progress: - **repetitive tool fingerprints** (`toolName` + primary-arg detail in a sliding window), or - **elevated ignored step-update rebuffs** (≥ 10) - Wire tool name/detail from `AgentLogger` → executor / step-session into `recordActivity(...)` so novelty is measurable. - Document the thrash-evidence rule in `docs/architecture.md`. ## Test plan - [x] `pnpm --filter @fusion/engine exec vitest run src/__tests__/stuck-task-detector.test.ts src/__tests__/reliability-interactions/non-progress-churn.test.ts` - [x] Regression: high-volume **diverse** iterative activity (174 events) does **not** classify as loop - [x] High bare text/heartbeat volume without tools does **not** classify as loop - [x] Repetitive identical tool fingerprint + timeout **does** classify as loop - [x] Ignored step-update thrash (≥10) with volume **does** classify as loop - [x] Existing FN-5168 no-progress-churn + FN-6598 verification suppression paths still pass - [ ] CI gate green <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved stuck/loop classification by requiring explicit “thrash evidence” (repetitive tool fingerprints and/or elevated ignored progress rebuffs), reducing false positives for busy but diverse work. * Updated loop evidence tracking to incorporate tool name plus summarized tool-argument detail. * Cleared loop evidence appropriately after verification, progress updates, and task resumption. * Extended tool-start telemetry/callbacks to include optional tool detail. * **Documentation** * Refined loop-classification criteria to match the new evidence gates. * **Tests** * Updated/expanded stuck/loop and churn scenarios to validate the evidence-based behavior and callback ordering. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
eef5eb751e |
FN-8453: unify concurrency accounting and indicators
Unify live-agent capacity accounting across engine and dashboard. - Derive Running and Waiting from workflow traits and durable agent liveness. - Apply unified limits to planner, executor, and merge admission while updating dashboard indicators. - Remove duplicate concurrency controls and document the unified operator model. Files changed: .changeset/fn-8453-unified-concurrency.md | 7 + docs/agent-tool-surface-full-loop.md | 4 +- docs/architecture.md | 2 +- docs/dashboard-guide.md | 4 +- docs/settings-reference.md | 4 +- .../skill/fusion/references/fusion-capabilities.md | 4 +- .../core/src/__tests__/live-agent-count.test.ts | 91 ++++---- packages/core/src/index.gate.ts | 6 + packages/core/src/index.ts | 6 + packages/core/src/live-agent-count.ts | 107 ++++++--- packages/dashboard/app/App.tsx | 28 ++- packages/dashboard/app/api/board-workflows.ts | 2 + packages/dashboard/app/components/Column.tsx | 6 +- .../dashboard/app/components/EngineControlMenu.tsx | 26 --- .../dashboard/app/components/ExecutorStatusBar.tsx | 38 ++- .../dashboard/app/components/SettingsModal.tsx | 1 - .../app/components/__tests__/Column.test.tsx | 6 +- .../__tests__/EngineControlMenu.test.tsx | 10 +- .../__tests__/ExecutorStatusBar.test.tsx | 32 ++- .../command-center/CommandCenterControls.tsx | 26 --- .../settings/sections/SchedulingSection.search.ts | 9 - .../settings/sections/SchedulingSection.tsx | 13 -- .../app/hooks/__tests__/useExecutorStats.test.ts | 12 +- packages/dashboard/app/hooks/useExecutorStats.ts | 50 ++-- .../src/__tests__/project-store-resolver.test.ts | 11 +- packages/dashboard/src/project-store-resolver.ts | 14 +- .../register-config-mcp-pi-settings-routes.ts | 3 +- packages/engine/src/__tests__/concurrency.test.ts | 123 +++++++++- .../engine/src/__tests__/project-engine.test.ts | 34 +++ packages/engine/src/__tests__/triage.test.ts | 7 +- packages/engine/src/concurrency.ts | 207 ++++++++++++++++- packages/engine/src/project-engine.ts | 151 ++++++++++-- packages/engine/src/scheduler.ts | 82 ++++++- packages/engine/src/triage.ts | 254 +++++++++++++-------- .../lib/dashboard-browser-safe-core-modules.json | 5 + 35 files changed, 991 insertions(+), 394 deletions(-) Fusion-Task-Id: FN-8453 Fusion-Task-Lineage: 12cfa5df-675d-4fce-b17e-932376544239 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
104bf69b3b |
FN-8401: preserve same-agent duplicates across create paths
Unify same-agent duplicate intake so live duplicates remain visible and sticky tombstones block recreation on every backend. - Route SQLite and backend creation through one duplicate-intake resolver - Flag new live duplicates by default; archive only the new task when explicitly enabled - Include archived soft-deletes in sticky tombstone matching and cover the backend-safe read - Document the cross-backend duplicate and resurrection policy Files changed: .changeset/fn-8401-same-agent-intake.md | 7 + docs/architecture.md | 2 +- docs/settings-reference.md | 2 +- docs/task-management.md | 8 +- .../__tests__/same-agent-duplicate-intake.test.ts | 129 ++++++++++++ packages/core/src/task-store/remaining-ops-2.ts | 50 +---- packages/core/src/task-store/task-creation.ts | 220 ++++++++------------- 7 files changed, 233 insertions(+), 185 deletions(-) Fusion-Task-Id: FN-8401 Fusion-Task-Lineage: 2efa998e-a27d-4013-b42e-e44b7e2316fb Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
93437ff09b |
FN-8306: add mission symbol lock scheduler admission
Schedule approved mission work with durable symbol-level concurrency control. - Gate mission execution on active lineage and required plan approval - Acquire, renew, and release symbol locks across scheduler and workflow execution - Honor custom workflow WIP columns when maintaining symbol-lock leases - Add admission, contention, renewal, and custom-WIP regression coverage Files changed: .../fn-8306-mission-symbol-scheduler-admission.md | 7 ++ docs/architecture.md | 1 + packages/core/src/task-store/moves.ts | 23 ++++ .../src/__tests__/mission-symbol-admission.test.ts | 54 +++++++++ .../__tests__/scheduler-workflow-cutover.test.ts | 49 ++++++++ .../src/__tests__/workflow-work-processor.test.ts | 39 ++++++ .../src/__tests__/workflow-work-scheduler.test.ts | 44 +++++++ packages/engine/src/mission-symbol-admission.ts | 94 +++++++++++++++ packages/engine/src/scheduler.ts | 133 ++++++++++++++++++++- packages/engine/src/workflow-work-processor.ts | 22 +++- packages/engine/src/workflow-work-scheduler.ts | 62 +++++++++- 11 files changed, 520 insertions(+), 8 deletions(-) Fusion-Task-Id: FN-8306 Fusion-Task-Lineage: ec6a9928-d8c0-4cf7-99b5-47e1fc3a2dcc Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
3008eb2dd8 |
FN-8405: add durable task symbol declarations
Persist normalized task symbol declarations and resolve symbols solely from durable task data. - Add declared-symbol parsing, normalization, and durable TaskStore resolution APIs. - Persist declarations through PostgreSQL migration, task serialization, and archive/restore flows. - Cover declaration precedence and schema upgrades with core tests. Files changed: docs/architecture.md | 1 + docs/storage.md | 1 + .../src/__tests__/postgres/schema-applier.test.ts | 39 +++- .../src/__tests__/task-symbol-resolution.test.ts | 222 +++++++++++++++++++++ packages/core/src/index.ts | 10 + .../core/src/postgres/migrations/0000_initial.sql | 1 + .../migrations/0028_task_declared_symbols.sql | 1 + packages/core/src/postgres/schema-applier.ts | 12 +- packages/core/src/postgres/schema/project.ts | 1 + packages/core/src/store.ts | 22 +- .../core/src/task-store/archive-lifecycle-2.ts | 2 + packages/core/src/task-store/persistence.ts | 4 +- packages/core/src/task-store/remaining-ops-2.ts | 2 +- packages/core/src/task-store/serialization.ts | 2 + packages/core/src/task-store/task-creation.ts | 5 + packages/core/src/task-store/task-row-mappers.ts | 2 +- packages/core/src/task-store/task-update.ts | 9 + packages/core/src/task-symbol-resolution.ts | 65 ++++++ packages/core/src/types.ts | 5 + 19 files changed, 400 insertions(+), 6 deletions(-) Fusion-Task-Id: FN-8405 Fusion-Task-Lineage: ca86dbe5-cd4c-4f58-b607-678d669875af Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
efa9580b93 |
FN-8366: enforce end-to-end project scoping
Ensure asynchronous dashboard data and real-time streams remain bound to their active project context. - Resolve SSE and badge WebSocket stores through the canonical project resolver - Guard agents, artifacts, and documents hooks against stale project responses and events - Add scoped cache handling, regression coverage, and architecture documentation Files changed: docs/architecture.md | 7 ++++ .../app/hooks/__tests__/useAgents.test.ts | 24 +++++++++++ .../app/hooks/__tests__/useArtifacts.test.ts | 47 ++++++++++++++++++++++ .../app/hooks/__tests__/useDocuments.test.ts | 34 ++++++++++++++++ packages/dashboard/app/hooks/useAgents.ts | 47 +++++++++++++++++----- packages/dashboard/app/hooks/useArtifacts.ts | 26 +++++++++--- packages/dashboard/app/hooks/useDocuments.ts | 18 ++++++++- .../dashboard/app/hooks/useProjectContextGuard.ts | 38 +++++++++++++++++ .../routes-context-project-identity.test.ts | 12 ++++++ packages/dashboard/src/server.ts | 21 +++++++--- 10 files changed, 251 insertions(+), 23 deletions(-) Fusion-Task-Id: FN-8366 Fusion-Task-Lineage: 0fcd38c1-727f-4511-8209-5d70cbf9eb0d Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
ccb7d4e8ff |
FN-8367: enforce bounded engine shellouts
Enforce bounded synchronous shellout use across the engine. - Audit every production synchronous shellout against a call-site allowlist. - Bound data-dependent git diff commands by timeout and output size. - Document the async shellout invariant and align focused command guards. Files changed: AGENTS.md | 2 +- docs/architecture.md | 1 + .../__tests__/engine-no-blocking-shellout.test.ts | 135 +++++++++++++++++++++ .../user-configured-command-no-execsync.test.ts | 5 +- packages/engine/src/merger-git-parse.ts | 16 ++- .../engine/src/merger-workspace-test-commands.ts | 27 ++++- 6 files changed, 181 insertions(+), 5 deletions(-) Fusion-Task-Id: FN-8367 Fusion-Task-Lineage: 976384e6-f283-4464-9f74-f328f2be3430 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
f85acce548 |
FN-8361: guard planning recovery mutations atomically
Keep delayed planning recovery mutations conditional on the task still being in its planning stage. - Add task-lock-backed conditional move and deletion APIs for planning recovery. - Guard triage and self-healing writes, including prompt-file mutations, with live planning predicates. - Cover conditional task operations and stale recovery behavior with regression tests. Files changed: docs/architecture.md | 2 +- .../src/__tests__/delete-task-if-planning.test.ts | 56 +++++++++++++ .../src/__tests__/move-task-if-planning.test.ts | 43 ++++++++++ packages/core/src/store.ts | 22 ++++- .../core/src/task-store/archive-lifecycle-2.ts | 35 ++++++++ packages/core/src/task-store/archive-lifecycle.ts | 65 +++++++++++++++ packages/core/src/task-store/moves.ts | 32 ++++++++ packages/engine/src/__tests__/self-healing.test.ts | 26 ++++++ packages/engine/src/__tests__/triage.test.ts | 14 +++- packages/engine/src/self-healing.ts | 30 ++++++- packages/engine/src/triage.ts | 95 ++++++++++++++++------ 11 files changed, 383 insertions(+), 37 deletions(-) Fusion-Task-Id: FN-8361 Fusion-Task-Lineage: d76f1805-d8c6-4b3f-9bc6-12907aae9733 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
3f7c32c95c |
refactor(cutover 2/3): engine — graph-owned lifecycle, legacy execution deleted (#2342)
Part **2 of 3** of the IR-driven lifecycle cutover (stacked on #2341; top is #2335). **Scope (80 files, packages/engine + cli/pi skill docs + AGENTS/architecture):** graph-driven column moves via the column-boundary controller (R1), single-mover scheduler/hold-release trait cutover (KTD-2/KTD-9), trait re-keyed self-healing + merger with the R7b confirmed-merge-must-finalize guarantee, graph-exclusive Plan Review with leased dedup (R4/R5), the executeCore body-lift — zero legacy re-entry — with fn_review_step + interceptor machinery deleted and tombstone-ratcheted (R9), builtin workflow runtime fixes (missing hold handler, unseamed-node column inheritance, no-merge completion mover), the 6-column benchmark acceptance suite (11 tests) + 12-builtin lifecycle sweep (94 assertions), and the executor test-harness modernization. Also retires core's interpreter-cutover scaffolding whose last consumer (the authoritative driver) dies here. **Merge order:** #2341 → this → #2335. After #2341 merges, retarget this to main. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0dbe67c851 |
FN-8356: clear stale duplicate decision pauses
Clear inactive duplicate markers so eligible tasks resume planning instead of showing a stranded decision badge. - Reconcile stale triage-marker duplicate pauses during self-healing and record audit events. - Clear inactive canonical markers during triage while preserving user and unrelated pauses. - Cover missing, deleted, completed, and archived canonical states with regression tests. Files changed: .changeset/fn-8356-stale-duplicate-decision.md | 7 ++ AGENTS.md | 1 + docs/architecture.md | 1 + .../explicit-duplicate-marker-sweep.test.ts | 43 ++++++-- .../self-healing-stale-duplicate-decision.test.ts | 109 +++++++++++++++++++++ .../triage-explicit-duplicate-marker.test.ts | 32 ++++-- packages/engine/src/run-audit.ts | 2 + packages/engine/src/self-healing.ts | 87 ++++++++++++++-- packages/engine/src/triage.ts | 41 ++++++-- 9 files changed, 298 insertions(+), 25 deletions(-) Fusion-Task-Id: FN-8356 Fusion-Task-Lineage: 8df8f0ee-d73e-41d6-8abe-a4b33662c9da Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
b61311baa8 |
FN-8305: add durable PostgreSQL symbol locks
Introduce durable project-scoped symbol locks backed by PostgreSQL. - Add normalized lease-based lock acquisition, renewal, release, and reconciliation APIs with audit events. - Add PostgreSQL schema migrations and self-healing reconciliation coverage. - Document the lock model and test migration and lock behavior. Files changed: AGENTS.md | 1 + docs/architecture.md | 1 + docs/storage.md | 7 + .../src/__tests__/postgres/schema-applier.test.ts | 115 +++++++++- packages/core/src/__tests__/symbol-locks.test.ts | 91 ++++++++ packages/core/src/index.ts | 17 ++ .../core/src/postgres/migrations/0000_initial.sql | 25 +++ .../src/postgres/migrations/0025_symbol_locks.sql | 63 ++++++ packages/core/src/postgres/schema-applier.ts | 18 +- packages/core/src/postgres/schema/project.ts | 29 +++ packages/core/src/store.ts | 23 ++ packages/core/src/symbol-lock-types.ts | 60 +++++ packages/core/src/task-store/symbol-locks.ts | 244 +++++++++++++++++++++ .../__tests__/symbol-lock-reconciliation.test.ts | 19 ++ packages/engine/src/self-healing.ts | 35 +++ 15 files changed, 745 insertions(+), 3 deletions(-) Fusion-Task-Id: FN-8305 Fusion-Task-Lineage: efd95c73-23e3-4359-8204-dfad374a39bc Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
5d2c3be6a0 |
FN-8221: clear inactive planner overseer state
Clear retained planner overseer state when effective oversight is disabled. - Remove monitor, recovery, advisor, and dedup runtime for oversight-off tasks. - Suppress stale oversight-off Eye badges in task cards. - Cover cleanup and badge behavior with regression tests. - Document the runtime snapshot invariant and add a patch changeset. Files changed: .changeset/fn-8221-overseer-badge-oversight-off.md | 7 ++ docs/architecture.md | 4 + packages/dashboard/app/components/TaskCard.tsx | 9 +- .../app/components/__tests__/TaskCard.test.tsx | 20 ++++ .../__tests__/planner-overseer-off-cleanup.test.ts | 119 +++++++++++++++++++++ packages/engine/src/project-engine.ts | 12 +++ 6 files changed, 170 insertions(+), 1 deletion(-) Fusion-Task-Id: FN-8221 Fusion-Task-Lineage: 01c9d838-fbe4-4d34-8eb5-d735cf35e581 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
b687cc994e |
FN-8174: preserve live triage planning sessions
Keep active planning sessions protected from stale recovery while reclaiming genuinely hung triage work. - Retain stale processing entries that still have a live, non-aborted triage session. - Continue evicting no-session and stuck-aborted tasks so recovery can proceed. - Add triage and self-healing regression coverage, architecture guidance, and a patch changeset. Files changed: .changeset/fn-8174-planning-premature-todo.md | 7 ++ docs/architecture.md | 1 + packages/engine/src/__tests__/self-healing.test.ts | 102 +++++++++++++++++++++ packages/engine/src/__tests__/triage.test.ts | 37 +++++++- packages/engine/src/triage.ts | 48 +++++----- 5 files changed, 168 insertions(+), 27 deletions(-) Fusion-Task-Id: FN-8174 Fusion-Task-Lineage: f6811d72-95b4-4b5f-a71f-212f50e3ecdd Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
edc64138e0 |
feat(worktrees): task-pinned worktrees under worktreeNaming "task-id" (#2233)
## Summary Adds **task-pinned worktrees** for `worktreeNaming: "task-id"`. Under task-id naming, a task is pinned to exactly one derivable directory `<worktreesDir>/<lowercased-task-id>` (e.g. `.worktrees/fn-7996`) for its entire lifecycle — removing the ambiguity that let stale/foreign `task.worktree` pointers strand a task (the FN-7996 shape). `recycleWorktrees` stays fully functional and is **mutually exclusive** with task-id pinning: the two can't be enabled together. ## Behavior - **Pinned acquisition (`worktreeNaming: "task-id"`, recycling off):** `acquireTaskWorktree` runs **derive → validate → reuse-or-recreate** at the derived path — warm-reuse when the dir is a registered, usable worktree on the task's own branch; otherwise reclaim-in-place (`removeWorktree` + recreate at the SAME path, never a sibling name). A disagreeing `task.worktree` cache self-corrects and emits a new `worktree:pin-rederived` audit event, without consuming worktree-session retries. The recycle pool is never consulted in pinned mode. - **Mutual exclusivity:** enabling both `recycleWorktrees` and `worktreeNaming: "task-id"` is rejected at the settings-write boundary — HTTP 400 at `PUT /settings`, and an `Error` backstop in `store.updateSettings` covering the CLI and every other writer (`assertWorktreeNamingRecycleExclusive`). The runtime also gates pinned mode on `!recycleWorktrees`, so a legacy on-disk config carrying both degrades safely to recycling. - **Settings UI:** the Settings → Worktrees panel enforces the exclusivity bidirectionally — the *Recycle worktrees* toggle is disabled while naming is *Task ID*, and the naming select is disabled while recycling is on — so the conflicting state is unreachable, with help text explaining why. - **Byte-inert for the rest:** `random`/`task-title` naming and the recycle pool (incl. `merger.ts` release) are unchanged; worktrunk-managed layouts bypass pinning. ## Acceptance criteria (from the plan) 1. ✅ Pinned task dispatched N times only ever touches `<worktreesDir>/<task-id>` on its own branch 2. ✅ No code path can hand task A's dir to task B (pool bypassed; path derived from task id) 3. ✅ FN-7996 stale/foreign `task.worktree` self-corrects at next dispatch (`worktree:pin-rederived`) without consuming session retries 4. ✅ Non-pinned modes with `recycleWorktrees: true|false` are byte-identical (existing pool tests pass unchanged) 5. ✅ Stale same-name dir (crash leftover / archive→restore) reclaimed in place, never suffixed 6. ✅ Docs updated (settings-reference, architecture, `worktreeNaming` type doc); changeset (`minor`, `feature`); FNXC comments encode the invariant ## Files - `packages/engine/src/worktree-pinning.ts` — new pure helpers (`isTaskPinnedWorktreeNaming`, `pinnedWorktreePathForTask`) - `packages/engine/src/worktree-acquisition.ts` — pinned branch + branch-match reclaim-in-place - `packages/engine/src/run-audit.ts` — `worktree:pin-rederived` audit type - `packages/core/src/settings-validation.ts` (+ `index.ts`, `task-store/settings-ops.ts`) — mutual-exclusion validator + wiring - `packages/dashboard/src/routes/register-settings-memory-routes.ts` — 400 on conflict - `packages/dashboard/app/components/settings/sections/WorktreesSection.tsx` (+ `packages/i18n/locales/en/app.json`) — bidirectional UI exclusivity - `packages/core/src/types.ts`, `docs/*`, `.changeset/*` ## Verification - New tests: engine `worktree-pinning` (5) + `worktree-acquisition-pinned` (7); core `worktree-naming-recycle-exclusive` (2); dashboard settings-route 400 (3) + WorktreesSection UI exclusivity (3) - Regression sweep green: 194 engine worktree/acquisition/pool/executor/merger-release tests, core settings tests, dashboard i18n/settings-section tests - `tsc --noEmit` clean for `@fusion/core` and `@fusion/engine`; changed source files clean; eslint clean - `pnpm verify:fast` PASS (build + scoped typecheck + boot smoke) 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Task ID worktree naming, providing each task with a stable, deterministic worktree directory. * Automatically reuses valid pinned worktrees and recreates stale or conflicting ones at the same path. * Added clear settings controls and validation for incompatible Task ID naming and worktree recycling options. * **Documentation** * Updated worktree architecture, settings reference, and in-app guidance to explain pinned worktrees and configuration constraints. * **Bug Fixes** * Improved recovery from stale or incorrect worktree assignments without consuming session retries. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
2c3a777bca |
FN-8132: recover bare worktree branch collisions
Recover safe worktree creation when a dangling task branch already exists. - Classify bare branch collisions and preserve foreign or mixed commit history. - Reuse task-owned branches or recreate merged branches from the pinned start point. - Audit recovery outcomes and cover native, fallback, and workspace acquisition paths. Files changed: .../fn-8132-worktree-branch-collision-recovery.md | 7 ++ docs/architecture.md | 1 + .../__tests__/worktree-acquisition-backend.test.ts | 69 +++++++++++ .../worktree-acquisition-workspace.test.ts | 21 ++++ .../worktree-backend-branch-collision.test.ts | 132 +++++++++++++++++++++ packages/engine/src/branch-conflicts.ts | 121 +++++++++++++++++++ packages/engine/src/run-audit.ts | 1 + packages/engine/src/worktree-acquisition.ts | 3 +- packages/engine/src/worktree-backend.ts | 94 +++++++++++++++- 9 files changed, 447 insertions(+), 2 deletions(-) Fusion-Task-Id: FN-8132 Fusion-Task-Lineage: f09d140f-4fcd-48a6-99b1-a351630f37bd Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
60b6e3e048 |
FN-7996: add configurable executor tool-failure retries
Add bounded, durable same-model retry handling for qualifying consecutive executor tool errors. - Persist retry claims, cursors, and audit markers with PostgreSQL migrations. - Expose project retry count, backoff, and failure threshold settings in the dashboard. - Cover retry, exhaustion, reset, and stale-run safety behavior with tests. Files changed: .changeset/fn-7996-executor-tool-failure-retry.md | 7 + AGENTS.md | 1 + docs/architecture.md | 1 + docs/settings-reference.md | 10 ++ .../executor-tool-failure-retry-claim.test.ts | 17 +++ .../core/src/__tests__/manual-retry-reset.test.ts | 3 + .../core/src/__tests__/settings-defaults.test.ts | 15 +- packages/core/src/in-review-stall.ts | 20 +++ packages/core/src/index.gate.ts | 6 + packages/core/src/index.ts | 6 + packages/core/src/manual-retry-reset.ts | 3 + .../0013_executor_tool_failure_retry.sql | 4 + packages/core/src/postgres/schema-applier.ts | 17 +++ packages/core/src/postgres/schema/project.ts | 3 + packages/core/src/settings-schema.ts | 3 + packages/core/src/store.ts | 10 +- packages/core/src/task-store/persistence.ts | 7 + packages/core/src/task-store/remaining-ops-2.ts | 2 +- packages/core/src/task-store/remaining-ops-3.ts | 2 +- packages/core/src/task-store/remaining-ops-6.ts | 65 ++++++++- packages/core/src/task-store/serialization.ts | 3 + packages/core/src/task-store/task-update.ts | 6 + packages/core/src/types.ts | 16 +++ .../dashboard/app/components/SettingsModal.tsx | 15 ++ .../app/components/settings/section-keys.ts | 3 + .../settings/sections/SchedulingSection.search.ts | 27 ++++ .../settings/sections/SchedulingSection.tsx | 4 + .../settings-default-descriptions.test.tsx | 3 + .../__tests__/executor-tool-failure-retry.test.ts | 160 +++++++++++++++++++++ packages/engine/src/executor.ts | 87 ++++++++++- packages/i18n/locales/en/app.json | 6 + 31 files changed, 523 insertions(+), 9 deletions(-) Fusion-Task-Id: FN-7996 Fusion-Task-Lineage: d1682ef8-534c-410e-b74c-1f2cf176eac2 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
c2475b012d |
FN-8064: add proactive task chat status updates
Task-detail chat now narrates engine progress and review outcomes in real time. - Emit bounded, redacted status rows for step lifecycle and review paths. - Present status entries with a distinct task-chat treatment. - Cover status narration and diagnostic sanitization with engine tests. Files changed: .changeset/fn-8064-proactive-chat.md | 7 + docs/architecture.md | 1 + packages/dashboard/app/components/TaskChatTab.css | 16 ++ packages/dashboard/app/components/TaskChatTab.tsx | 9 +- .../engine/src/__tests__/executor-prompt.test.ts | 28 +++- .../engine/src/__tests__/proactive-status.test.ts | 54 +++++++ packages/engine/src/executor.ts | 176 ++++++++++++++++----- packages/engine/src/proactive-status.ts | 117 ++++++++++++++ 8 files changed, 365 insertions(+), 43 deletions(-) Fusion-Task-Id: FN-8064 Fusion-Task-Lineage: c6d0a9b5-0946-4bf4-8338-e982e1cbfd53 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
402b3a91fa |
fix(FN-8004): treat heartbeat soft-delete races as benign instead of stranding agents
A task soft-deleted concurrently with a heartbeat-driven moveTask raised TaskDeletedError from the engine's own board path, leaving the agent in `error` with a non-empty lastError and requiring a stop/start cycle to recover. The race is benign by construction: the task is gone, so the move is a no-op. The heartbeat now classifies it via isConcurrentSoftDeleteRaceError (matching the canonical message and serialized/typed forms), keeps the agent active, clears stale error/recovery state, and emits agent:heartbeat-move-skipped-soft-delete with ids/counts-only metadata. Concurrent operator pauses are preserved. Squash-merged by hand from fusion/fn-8004. The engine's AI merge approved this content twice (squash a3a3cc6a8) but could not land it: main advances every ~8 minutes and each merge cycle took ~10, so every attempt lost to a concurrent advance and rebuilt. Each cycle also burned a corrective pass on a first-pass review rejection with no stated reason — the issue #1946 class of bug that this task's own report cites as a sibling. Reconciled against #2157, which refactored transient-error-detector.ts: the new classifier coexists with the extracted transient-error-patterns.ts leaf. Verified on the merged tree — 123 tests green across FN-8004's suites and #2157's, engine typecheck clean. Fusion-Task-Id: FN-8004 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
08a10bf486 |
fix(FN-8006): back off and pause Plan Review on provider rate limits
A rate-limited Plan Review re-ran every 30s for hours (~1,900 requests
per 5h window, reviewerFallbackRetryCount observed past 100), which is
the request volume that trips a provider's low-interactivity throttle —
so the retry storm prolonged the very outage it was retrying.
Root cause: runPlanReviewBeforeExecution catches every reviewStep throw
inline to keep triage alive, which converts them all to an UNAVAILABLE
verdict. That laundering had two consequences the earlier fixes missed:
FN-8006 terminalized RetryStormError and the reviewer started throwing
ReviewerProviderError for 429s, but a ReviewerProviderError still landed
in the UNAVAILABLE park — a FIXED 30s nextRecoveryAt with no attempt
counter and no cap. The reviewer's own escalation contract ("escalate so
UsageLimitPauser pauses every lane") held only on the executor path,
because the inline catch hid the error from triage's usage-limit handler
in specifyTask.
- triage: fire usageLimitPauser.onUsageLimitHit for usage-limit reviewer
failures, so a 429 pauses every lane instead of re-parking one task.
- triage: re-park via computeRecoveryDecision (60s/120s/240s, ±10%
jitter) and terminalize at MAX_RECOVERY_RETRIES. A reviewer that never
yields a verdict is a real failure and must surface, not spin.
- triage: clear the borrowed recoveryRetryCount budget on any real
verdict, so surviving an outage cannot shorten the executor's later
transient budget.
- core: RetryStormError takes an optional cause, surfaced as
underlyingError in serializeRetryStormError and folded into the
message, so a cap no longer masks the real error. recordRetry threads
it from the reviewer's error path.
Surface enumeration: the park is driven by a thrown provider error, a
thrown generic error, and a plain UNAVAILABLE verdict with no throw.
All three are covered — a repro pinned only to the reported 429 would
leave the other two spinning on the old fixed timer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
|
71dd191c7c |
FN-8006: terminalize Plan Review retry storms
Plan Review now fails tasks when reviewer fallback retry limits are exceeded. - Detect RetryStormError from Plan Review workflow execution - Serialize the terminal retry error, clear recovery scheduling, and preserve workflow results - Add retry-storm regression coverage, architecture guidance, and a patch changeset Files changed: .changeset/fn-8006-plan-review-retry-storm.md | 7 ++++ docs/architecture.md | 2 +- packages/engine/src/__tests__/triage-plan-review-unavailable-retry.test.ts | 47 +++++++++++++++++++++- packages/engine/src/triage.ts | 33 +++++++++++++++ 4 files changed, 87 insertions(+), 2 deletions(-) Fusion-Task-Id: FN-8006 Fusion-Task-Lineage: 932e7930-2069-4b0c-9cd1-9db39c2de5a3 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
cae7847085 |
fix(FN-8004): retry ACP provider blips in auto-merge instead of parking failed (#2157)
## What happened
FN-8004's implementation work finished and passed review. The auto-merge
then failed with `Grok ACP turn failed: Internal error` — a ~20 second
provider blip — and the task was parked `status: "failed"` with 8 files
of complete, reviewed work stranded on its branch.
The park is the interesting part: `status: "failed"` is precisely what
tells recovery to stop. So a misclassification here isn't a missed
retry, it's **terminal**. Both recovery paths were disabled by the same
wrong verdict:
- `maybeRetryTransientMerge` (inline, 3 retries w/ backoff) — never
fired once (`mergeTransientRetryCount: 0`).
- `recoverTransientMergeFailures` (self-healing sweep, exists exactly to
rescue parked in-review tasks) — skipped it, gated on the same
classifier.
## Three defects fixed
**1. No AI-provider failure class existed.** The AI merge drives a real
LLM turn, but `classifyTransientMergeError` only modeled git/lease/spawn
faults. Adds `ai-provider-turn-failure`.
**2. ACP dropped the error detail.** `promptAcpSession` rethrew the SDK
error unchanged, discarding the JSON-RPC `code`/`data` — the only
evidence the fault was provider-side. ("Internal error" is just the
standard text for `-32603`.) It now preserves them, keeping the original
as `cause`:
```
Internal error (acp rpc code -32603, retryable)
```
Classification anchors on that envelope, **not** on the bare `"Internal
error"` — matching that unanchored would disguise genuine application
defects as retryable blips. Only provider-fault codes (`-32603`,
`-32000`..`-32003`) are retryable; caller-fault codes
(`-32600`..`-32602`) stay permanent, since retrying just repeats the
failing call.
**3. Sweep/inline asymmetry** (found while tracing; latent and
unreported). The inline gate accepted `isTransientError(msg) ||
classify(msg)`, but the sweep consulted **only** the classifier. So
`ECONNRESET` / `socket hang up` during a merge earned inline retries and
then went **invisible to the sweep** once parked — stranded forever. The
classifier now delegates to `isTransientError`, so both gates agree by
construction.
To keep that delegation from importing the detector's
`usage-limit-detector → logger` chain (the chain FN-5627 split the
classifier out to avoid, which would break
`notification-service.test.ts`'s partial `vi.mock`), the pure predicates
moved to the import-free leaf `transient-error-patterns.ts`, re-exported
from `transient-error-detector.ts`. All 13 exports preserved, verified
programmatically.
## Loosened budgets
Per request, so more self-heals. Both apply **only** to errors already
proven transient; the ceiling and
`merger:transient-failure-budget-exhausted` audit path remain.
| Budget | Before | After |
|---|---|---|
| `MAX_AUTO_MERGE_TRANSIENT_RETRIES` | 3 | 5 (backoff
5s/10s/20s/40s/80s) |
| `MAX_TRANSIENT_MERGE_RECOVERIES` | 2 | 5 |
The bump broke two suites that had hardcoded the old `3`. Rather than
swap in another magic number, both now derive the cap from the constant
so future tuning doesn't re-break them.
## Verification
- `pnpm test:gate` green · `pnpm lint` clean · engine + ACP typecheck
clean · `pnpm verify:fast` PASS (5/5)
- ACP plugin 230 tests green · Grok plugin 64 green · engine
transient/merge suites 136 green
- Regression tests assert the **invariant across every surface** (per
*Fix the Invariant, Not the Repro*), not just the reported Grok string:
both ACP runtime prefixes, all retryable/non-retryable rpc codes, both
SDK error shapes, network delegation, class-ordering, and negative cases
proving bare `"Internal error"` and real defects stay permanent.
- A test caught a genuine bug in my own code mid-review (nested-shape
message shadowing), now fixed.
- `notifier.test.ts > "awaiting approval"` fails — **confirmed
pre-existing on clean main**, unrelated.
## Note
FN-8004's own branch (`fusion/fn-8004`) is still unmerged and its work
looks complete. Once this lands, its merge should be retried separately.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
|
214af98591 |
FN-7977: hold Plan Review provider failures without replan regression
Prevent provider, model, transport, and abort failures from bouncing tasks back to planning after they enter execution. - Classify non-plan-defect Plan Review failures and skip needs-replan handoff - Terminate graph traversal with plan-review-provider-failure-hold and retry in place - Guard triage recovery so advanced column/worktree/step state is never overwritten - Document planning-recovery no-regression invariant and add regression tests - Add patch changeset for the operator-facing fix Files changed: .changeset/fn-7977-planning-failure-no-regression.md | 7 ++ docs/architecture.md | 1 + docs/workflow-steps.md | 2 +- packages/engine/src/__tests__/replan-target.test.ts | 17 +++- packages/engine/src/__tests__/transient-error-detector.test.ts | 32 +++++- packages/engine/src/__tests__/triage.test.ts | 110 +++++++++++++++++++++ packages/engine/src/__tests__/workflow-graph-optional-group.test.ts | 46 ++++++++- packages/engine/src/__tests__/workflow-graph-optional-step-fix.test.ts | 36 +++++++ packages/engine/src/executor.ts | 62 +++++++++++- packages/engine/src/replan-target.ts | 22 +++++ packages/engine/src/transient-error-detector.ts | 37 +++++++ packages/engine/src/triage.ts | 73 +++++++++++--- packages/engine/src/workflow-graph-executor.ts | 45 ++++++++- 13 files changed, 466 insertions(+), 24 deletions(-) Fusion-Task-Id: FN-7977 Fusion-Task-Lineage: 6d62d3ca-c6f3-4d02-a377-d7fd59f0c0f9 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
85f8b1f909 |
feat: shared Postgres multi-node — retire mesh data-plane replication (#2130)
## Summary - Treat **shared PostgreSQL** (`DATABASE_URL`) as the multi-node durable data plane; mesh HTTP is membership + optional auth, not task/settings replication. - **Peer exchange**: under Postgres backend mode, write queue is **topology/auth-only**; non-topology pending rows fail rather than replaying multi-leader task/settings payloads. - **Mesh routes**: task-ID reserve/commit/abort always hit local shared allocator rows (ignore remote `coordinatorNodeId`); mesh sync ignores settings and only exchanges `authMaterial`. - **Docs**: rewrite multi-project runbook, shared cluster protocol, and architecture mesh sections for shared-Postgres + claims/leases. ## Context Follows the SQLite→Postgres cutover. Multiple Fusion nodes can share one external Postgres while keeping **per-node execution** (worktrees, processes, claims via `central.task_claims`). Explicit non-goals remain: scheduler failover and live process migration. Plan: `docs/plans/2026-07-15-001-refactor-mesh-shared-postgres-multinode-plan.md` ## Test plan - [x] `pnpm --filter @fusion/engine exec vitest run src/__tests__/peer-exchange-service.test.ts` - [x] `pnpm --filter @fusion/dashboard exec vitest run src/__tests__/mesh-routes.test.ts` - [x] `pnpm --filter @fusion/core exec vitest run src/__tests__/shared-mesh-state.test.ts` - [ ] CI gate (lint/typecheck/build/gate) - [ ] Manual (optional): two processes, same `DATABASE_URL`, create task on A visible on B; settings change without mesh settings sync; claim exclusivity ## Operator note Multi-node shared board requires **external** `DATABASE_URL` on every node. Default embedded Postgres is still single-host. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Improved multi-node deployments using shared PostgreSQL as the durable source of execution state. * Task ID reservation/commit/abort now run locally (no remote coordinator forwarding). * Mesh syncing now prioritizes topology visibility and authentication material; settings replication is disabled in shared-Postgres mode. * **Bug Fixes** * Prevented task/settings replication over mesh HTTP in shared-Postgres deployments. * Refined lease ownership, recovery, and reconciliation to converge via shared-database primitives. * **Documentation** * Updated architecture and shared-mesh protocol guidance, including multi-node setup and lease/task-ID allocation behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
6e3a338cac |
FN-7968: defer slow cleanup off task deletion critical path
Make soft-delete return after the DB mutation while branch and agent cleanup run in the background. - Schedule cleanupBranchForTask after the soft-delete transaction instead of awaiting it under withTaskLock - Persist cleaned-branch log entries on the deleted row asynchronously; warn on deferred failures - Respond from DELETE /tasks/:id after deleteTask and schedule execution-agent binding release off the HTTP path - Add core and dashboard regression tests for non-blocking delete cleanup - Document the fast-path contract in architecture.md and add a patch changeset Files changed: .changeset/fn-7968-task-delete-latency.md | 7 + docs/architecture.md | 1 + .../task-delete-nonblocking-cleanup.test.ts | 160 +++++++++++++++++++++ packages/core/src/task-store/archive-lifecycle.ts | 57 +++++++- .../routes-task-delete-nonblocking.test.ts | 139 ++++++++++++++++++ .../src/routes/register-task-workflow-routes.ts | 19 ++- 6 files changed, 370 insertions(+), 13 deletions(-) Fusion-Task-Id: FN-7968 Fusion-Task-Lineage: f218a91e-aee3-46c9-a80f-182751b3ccc4 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
a242f1b449 |
fix(FN-7952): migrate bundled plugins to PostgreSQL (#2111)
## Summary Bundled plugins now persist shared runtime state in project-scoped PostgreSQL tables instead of maintaining independent SQLite authority. Reports, CLI Printing Press, Compound Engineering, Roadmap, Even Realities, and WhatsApp all follow the same ownership and startup contract as Fusion core. ## Design decisions - Plugin schema hooks run through the host’s PostgreSQL owner and enforce project isolation. - The SDK exposes the host contract needed by bundled plugins without importing engine internals. - Legacy Roadmap ownership fixtures use the supported empty-owner sentinel, preserving current composite primary/foreign keys while exercising backfill behavior. - The lockfile travels with the Even Realities PostgreSQL dependency so packaged installs remain reproducible. ## Validation - All six affected plugin builds pass. - Affected plugin suites pass: 773 tests across Printing Press, Compound Engineering, Even Realities, Reports, Roadmap, and WhatsApp. - `pnpm test:gate` passes all 478 gate tests. - This PR changes 40 files. ## Stack - Depends on #2110 → #2109 → #2108. - The documentation/release PR completes the stack. Related: #2105 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Breaking Changes** * PostgreSQL is now required for runtime storage; SQLite files are used only as one-time migration inputs. * The legacy `FUSION_NO_EMBEDDED_PG` fallback has been removed. * **New Features** * Added project-isolated PostgreSQL storage for plugins, reports, tasks, notifications, and other plugin data. * Added agent tools for reports and CLI service drafts. * Added PostgreSQL schema initialization support for plugin authors. * **Bug Fixes** * Improved migration and recovery of legacy plugin state. * Prevented cross-project data access and strengthened transactional schema updates. * **Documentation** * Updated storage, migration, deployment, plugin authoring, CLI, and dashboard guidance for PostgreSQL. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
4f037679ad |
feat: planner overseer session advisor (OMP advisor parity) (#2082)
## Summary Adds a **session advisor** to the planner overseer so Fusion can review live executor transcripts the way [oh-my-pi’s advisor](https://github.com/can1357/oh-my-pi/tree/main/packages/coding-agent/src/advisor) does — without replacing the existing lifecycle supervisor (stage watch, retry, merge confirmation, human-control withhold). ### What ships - **Emission guard** (`OverseerEmissionGuard`) — content-free phrase filter, session dedupe with severity-rank escalation, one accept per advisor update - **Session delta runtime** — queues agent-log deltas, drains through an advisor agent, drops backlog after 3 failures - **Session advisor service** — model gate, level matrix (`observe` / `steer` / `autonomous`), human-control re-check at inject, `[session-advisor]` steering comments - **OVERSEER.md / WATCHDOG.md** discovery for project review priorities - **AgentLogger `onEntriesFlushed`** + poll-backed agent-log cursor for durable deltas - Workflow settings: `plannerOverseerAdvisorProvider` + `plannerOverseerAdvisorModelId` (both required; empty = soft-disabled for cost safety) - Docs + changeset ### What does not ship (deferred) - Multi-advisor YAML roster, mutating advisor tools, reviewer/merger shadowing, true tool-abort interrupt ### Plan `docs/plans/2026-07-13-001-feat-overseer-advisor-parity-plan.md` ## Enablement 1. Set workflow **Session advisor model provider** + **Session advisor model id** 2. Oversight level `observe` (log only), `steer`, or `autonomous` (inject) 3. Optional: add `OVERSEER.md` or `WATCHDOG.md` in the project ## Test plan - [x] `pnpm --filter @fusion/core exec vitest run src/__tests__/overseer-emission-guard.test.ts` - [x] `pnpm --filter @fusion/engine exec vitest run` overseer-* unit tests (21 tests) - [x] Related planner-overseer / intervention regression tests - [x] `@fusion/engine` + `@fusion/core` typecheck - [ ] Manual: configure advisor model, run an executor task, confirm `[session-advisor]` inject + timeline metadata when concern is raised ## Residual Review Findings None from autofix pass (log-cursor ordering fix already committed). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an off-by-default “session advisor” that can review live execution activity and provide severity-based guidance. * Added project and per-task controls to enable it, including a default enable switch and Quick Add / Task Detail toggles. * Enhanced advisor prompting by discovering and incorporating `OVERSEER.md`/`WATCHDOG.md` review files. * **Documentation** * Added architecture and settings documentation for the new session-advisor parity behavior. * **Bug Fixes** * Improved fail-soft handling so advisor behavior won’t disrupt execution. * Fixed concurrent PostgreSQL migration startup failures. * **Tests** * Added coverage for advice parsing, emission guarding, runtime behavior, and watchdog discovery. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
6e0fde860c |
FN-7949: fix deleted planning-mode session resurrecting after in-flight generation completes
Adds a bounded-TTL delete tombstone to AiSessionStore so a straggling post-delete generation write can never resurrect a session the user explicitly deleted. - AiSessionStore now records a 10-minute delete tombstone (id -> deletion timestamp) in delete(), deleteByIdAndType(), and bulk cleanup paths (cleanupOld/cleanupStaleSessions/emitDeletedSessions). - upsert() checks the tombstone first and drops (no-ops) any write for a tombstoned id without touching SQLite or emitting ai_session:updated, fixing the root cause once in the shared store rather than per-producer (planning.ts, subtask-breakdown.ts, mission-interview.ts, milestone-slice-interview.ts). - Tombstone entries are pruned lazily on check and piggyback on the existing cleanupStaleSessions() cadence so the in-memory map cannot grow unbounded. - Adds a changeset (patch) documenting the user-facing fix. - Updates docs/architecture.md and docs/storage.md with the new "AI session delete tombstones" behavior. - Adds regression tests covering the tombstone guard in ai-session-store.test.ts and routes-planning.test.ts. Files changed: .changeset/fn-7949-ai-session-delete-tombstone.md | 7 + docs/architecture.md | 2 +- docs/storage.md | 12 +- packages/dashboard/src/__tests__/ai-session-store.test.ts | 145 +++++++++++++++ packages/dashboard/src/__tests__/routes-planning.test.ts | 200 ++++++++++++++++++++- packages/dashboard/src/ai-session-store.ts | 83 +++++++++ 6 files changed, 446 insertions(+), 3 deletions(-) Fusion-Task-Id: FN-7949 Fusion-Task-Lineage: 8e509dae-0cc5-46cd-9c4b-9048cfda56d3 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
e35620c9aa |
FN-7939: supervise heartbeat timer-audit interval and bound non-advancing zombie re-arms
Fixes agents silently going stale for hours even though the heartbeat repair audit process was running. - HeartbeatTriggerScheduler now runs an independent watchdog (armTimerAuditWatchdog/checkTimerAuditLiveness) that tracks the audit loop's last-run timestamp and re-arms + immediately re-runs the 60s audit interval if it goes stale beyond a bounded multiple of the cadence, so a silently dropped audit driver self-heals instead of leaving active agents unrepaired for hours. - Tracks consecutive non-advancing zombie-timer re-arms per agent (nonAdvancingRearmState) and escalates once the count crosses a threshold, recording consecutiveNonAdvancingRearms/nonAdvancingEscalated in agent.metadata.heartbeatTimerRepair and logging reason=heartbeat-rearm-nonadvancing-escalated instead of silently churning the same zombie-timer-rearmed repair forever. - Clears non-advancing rearm state on unregister, non-eligible agents, paused settings, and stale-run-reap skip paths so tracking never leaks stale per-agent counters. - Watchdog and its interval handle are armed in start() and cleared in stop() alongside the existing audit interval. - Adds a changeset (patch) describing the fix, and updates docs/agents.md and docs/architecture.md to document the FN-7939 audit watchdog and non-advancing escalation behavior. - Adds heartbeat-scheduler.test.ts coverage for watchdog re-arm/liveness and non-advancing escalation. Files changed: .changeset/fn-7939-heartbeat-audit-supervision.md | 7 + docs/agents.md | 8 +- docs/architecture.md | 1 + .../src/__tests__/heartbeat-scheduler.test.ts | 209 +++++++++++++++++++++ packages/engine/src/agent-heartbeat.ts | 128 ++++++++++++- 5 files changed, 341 insertions(+), 12 deletions(-) Fusion-Task-Id: FN-7939 Fusion-Task-Lineage: 9fa90240-4333-4588-b595-aef3811b1524 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
316d4fa034 |
FN-7941: anchor execute-requeue loop guard to monotonic terminal-step progress
Hardens the FN-7863 execute-node self-requeue loop guard so residual execute_loop_stall cases (#2043/#2045/#2046/#2047) can no longer reset the loop counter forever via non-terminal signature drift. - Change buildExecuteRequeueLoopSignature to track terminal step count (done/skipped) plus total step count instead of raw currentStep + every step status, so pending/in-progress oscillation no longer produces a "new" signature each cycle. - Add buildExecuteRequeueLoopHighWaterSignature, which derives current terminal-step progress via the shared signature parser (parseExecuteRequeueLoopProgressSignature) and only resets the streak on monotonic forward progress, keeping a high-water mark across cycles so decreases/oscillation below the high-water still count toward exhaustion. - Update executor.ts's execute self-requeue dispatch path to use the new high-water helper when deciding whether to reset (1) or increment executeRequeueLoopCount, replacing the previous raw signature-equality check. - Extend execute-requeue-loop-guard.test.ts with regression coverage: a drifting-signature case that oscillates step order/status with no terminal progress (still terminalizes at MAX_EXECUTE_REQUEUE_LOOP_CYCLES), a done/in-progress oscillation case bounded after the high-water stops increasing, and an updated "real progress never terminalizes" case driven by genuine monotonic done-step advancement. - Update docs/architecture.md's FN-7863/FN-7926 self-healing notes to describe the new terminal-step high-water signature and cross-reference FN-7941. Files changed: docs/architecture.md | 4 +- .../execute-requeue-loop-guard.test.ts | 83 +++++++++++++++++++++- packages/engine/src/executor.ts | 54 ++++++++++++-- 3 files changed, 130 insertions(+), 11 deletions(-) Fusion-Task-Id: FN-7941 Fusion-Task-Lineage: cbf1e536-d29b-40da-bdd8-8c34d8d6b1ca Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
6dcecb0c34 |
FN-7926: park completed-but-blocked tasks instead of looping execute-requeue
Stops the execute → pause-abort → re-queue-to-todo infinite loop for tasks whose implementation work is done but a dependency/blockedBy blocker is still live, by diverting them into a dedicated parked state instead of feeding the FN-7863 no-progress backstop or looping forever. - Add TaskExecutor.parkCompletedBlockedTask(): when work is complete but getTaskCompletionBlocker() still reports a blocker, park the task in todo with pausedReason:"completed-work-blocked", status:"queued", preserved worktree/branch/steps, and a cleared execute-requeue signature. - Replace shouldFinalizeCompletedTask's boolean with getCompletedTaskFinalizationDecision() returning "finalize" | "blocked" | "incomplete" so both the paused-after-completion and finalization call sites can react to the new "blocked" outcome without re-entering execution. - Divert completed-but-blocked tasks before the FN-7863 execute-requeue-loop counter increments, so waiting-on-dependency states are no longer misclassified as EXECUTION_DISPATCH_LOOP_EXHAUSTED. - Add SelfHealingManager.reconcileCompletedBlockedTasks(): a bounded sweep (wired into both startup/maintenance and periodic self-healing passes) that clears the park and advances the task to review once getTaskCompletionBlockerForStore() resolves, guarded by auto-merge eligibility, user-pause, and live-execution checks; failed advances re-park rather than strand the row. - Add run-audit mutation types task:completed-blocked-parked and task:completed-blocked-advanced (ids/counts/outcomes-only metadata) plus AGENTS.md/docs/architecture.md entries documenting the new lifecycle. - Extend execute-requeue-loop-guard.test.ts with coverage for the park/advance flow, including the zero-step task edge case. Files changed: AGENTS.md | 1 + docs/architecture.md | 2 + .../execute-requeue-loop-guard.test.ts | 256 ++++++++++++++++++++- packages/engine/src/executor.ts | 85 ++++++- packages/engine/src/run-audit.ts | 4 + packages/engine/src/self-healing.ts | 95 ++++++++ 6 files changed, 432 insertions(+), 11 deletions(-) Fusion-Task-Id: FN-7926 Fusion-Task-Lineage: e47945f4-a816-447e-9ea1-7c13105d0ba9 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
2e7fce21ae |
FN-7884: reset durable-agent error state on engine restart
Engine startup now treats itself as an implicit operator retry for durable heartbeat agents stuck in error, clearing eligible error states and re-arming heartbeats instead of waiting for the steady-state sweep's cooldown/exhaustion gates. - Add SelfHealingManager.resetDurableAgentErrorStateOnStartup(), run first in runStartupRecovery(), which resets shared heartbeatErrorRecovery/legacy durableErrorRecovery metadata, clears lastError/pauseReason, flips eligible error and error-retry-exhausted-parked durable agents to active, and re-arms their heartbeat - Preserve suppression for operator-actionable, stale worktree/module-resolution, user-paused, error-unrecoverable, ephemeral, disabled-runtime, and actively-executing agents - Add agent:reset-error-state-on-startup run-audit mutation type with ids/counts/outcomes-only metadata (agentId, priorState, priorPauseReason, source) - Add changeset FN-7884 (patch) documenting the operator-facing behavior - Update AGENTS.md and docs/agents.md, docs/architecture.md to describe the new startup reset path alongside existing FN-7835/FN-7844/FN-7859/FN-7878 recovery docs - Extend self-healing.test.ts with coverage for the new startup reset behavior and its exclusions Files changed: .changeset/fn-7884-restart-error-reset.md | 7 ++ AGENTS.md | 1 + docs/agents.md | 4 +- docs/architecture.md | 2 +- packages/engine/src/__tests__/self-healing.test.ts | 127 ++++++++++++++++++++- packages/engine/src/run-audit.ts | 1 + packages/engine/src/self-healing.ts | 88 +++++++++++++- 7 files changed, 223 insertions(+), 7 deletions(-) Fusion-Task-Id: FN-7884 Fusion-Task-Lineage: fe64f6af-3ff3-4876-8308-8a75591c45f1 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
504dc69f02 |
FN-7878: default heartbeat error recovery to recoverable for generic durable-agent failures
Durable agents were parking as error-unrecoverable on any non-transient-pattern failure, even generic/unknown blips that manual Retry immediately fixed; this changes the default to recoverable and reserves immediate unrecoverable parking for operator-actionable errors. - isHeartbeatErrorRecoverable now returns true unless the error is operator-actionable (auth/model/billing/scope) or a stale worktree/module-resolution error, instead of requiring a transient-pattern match via classifyError - Add OAuth scope-requirement and insufficient-scope patterns to the operator-actionable error detector so those still park immediately - Update heartbeat-error-recovery, heartbeat-executor, self-healing, and transient-error-detector tests to cover the new default-recoverable behavior - Update AGENTS.md and docs/architecture.md durable-agent error recovery notes to describe the new recoverable-by-default policy - Add changeset documenting the fix Files changed: .changeset/fn-7878-recoverable-default.md | 7 ++ AGENTS.md | 2 +- docs/architecture.md | 4 +- .../src/__tests__/heartbeat-error-recovery.test.ts | 90 +++++++++++++++++++--- .../src/__tests__/heartbeat-executor.test.ts | 17 ++-- packages/engine/src/__tests__/self-healing.test.ts | 45 ++++++----- .../src/__tests__/transient-error-detector.test.ts | 7 +- packages/engine/src/agent-heartbeat.ts | 8 +- packages/engine/src/transient-error-detector.ts | 2 + 9 files changed, 137 insertions(+), 45 deletions(-) Fusion-Task-Id: FN-7878 Fusion-Task-Lineage: 6f929af9-ceef-404f-95c9-98f26478f020 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
9cfb40e137 |
FN-7863: add bounded execute-node self-requeue loop guard
Bounds the execute->pause-abort->todo dispatch loop so a task can no longer requeue forever with no visible signal or terminal state. - Track a progress-anchored `executeRequeueLoopCount`/`executeRequeueLoopSignature` pair on the task row (current step + step statuses) so slow no-progress requeue cycles are counted independently of the scheduler's wall-clock `dispatchStormCount` guard. - Warn visibly in the task log at `EXECUTE_REQUEUE_LOOP_VISIBLE_THRESHOLD` (3) and terminalize non-paused, non-terminal tasks at `MAX_EXECUTE_REQUEUE_LOOP_CYCLES` (6) with `status:"failed"` and an `EXECUTION_DISPATCH_LOOP_EXHAUSTED:` error, preserving worktree/branch/step progress. - Emit a new `task:execution-dispatch-loop-terminalized` run-audit mutation type with ids/counts/outcomes-only metadata. - Reset the loop counters on real progress, manual retry, forward moves (in-review/done/archived), and unpause, in both the executor and scheduler. - Add DB migration 142 (`executeRequeueLoopCount`, `executeRequeueLoopSignature` columns) plus store read/write/reset plumbing. - Add reliability-interactions coverage for the new loop guard and extend store-persistence tests for the new columns. - Document the new behavior in AGENTS.md and docs/architecture.md. Files changed: AGENTS.md | 1 + docs/architecture.md | 2 + packages/core/src/__tests__/store-persistence.test.ts | 45 +++++ packages/core/src/db.ts | 17 +- packages/core/src/manual-retry-reset.ts | 1 + packages/core/src/store.ts | 22 ++- packages/core/src/types.ts | 11 ++ .../execute-requeue-loop-guard.test.ts | 188 +++++++++++++++ packages/engine/src/executor.ts | 67 +++++++- packages/engine/src/run-audit.ts | 2 + packages/engine/src/scheduler.ts | 8 +- 11 files changed, 355 insertions(+), 9 deletions(-) Fusion-Task-Id: FN-7863 Fusion-Task-Lineage: db40507f-5851-435e-8854-c1ed695b4154 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
67cc025562 |
FN-7859: park non-recoverable durable heartbeat errors instead of stalling in bare error
Debug org agents error state recovery regression: durable heartbeat-managed agents with a non-recoverable error (permanent/credential/model-access/ config, not stale-worktree/module-resolution) were previously left indefinitely in bare `state:"error"` with no operator-visible reason, and CLI agent inspection tools did not surface error/pause diagnostics. - Timer path (`HeartbeatMonitor`) and run-entry recovery now classify non-recoverable durable heartbeat errors and park the agent `paused` with `pauseReason:"error-unrecoverable"` instead of restart-looping or sitting in `error` forever. - `SelfHealingManager` mirrors the same non-recoverable classification in its recovery sweep, parking with the same reason/metadata and skipping the exhausted/next-retry gates for that terminal bucket. - New `agent:error-parked-unrecoverable` run-audit event type emitted by both the heartbeat and self-healing paths (ids/counts/outcomes-only metadata). - `fn_agent_show` now prints `Last Error`, `Pause Reason`, and a compact `Error Recovery` counter line; `fn_list_agents` prints the same diagnostics only for agents currently in `error`/`paused`. - Updated `AGENTS.md`, `docs/agents.md`, and `docs/architecture.md` to document the new terminal-park behavior and CLI diagnostics surface. - Added a changeset (`@runfusion/fusion` patch) describing the operator-facing fix. Files changed: .changeset/fn-7859-org-agent-error-diagnostics.md | 7 ++ AGENTS.md | 2 +- docs/agents.md | 3 +- docs/architecture.md | 4 +- packages/cli/src/__tests__/extension.test.ts | 68 ++++++++++++++++ packages/cli/src/extension.ts | 64 +++++++++++++++ .../src/__tests__/heartbeat-error-recovery.test.ts | 47 ++++++++++- packages/engine/src/__tests__/self-healing.test.ts | 94 ++++++++++++++++++---- packages/engine/src/agent-heartbeat.ts | 71 +++++++++++++++- packages/engine/src/run-audit.ts | 1 + packages/engine/src/self-healing.ts | 46 +++++++++-- 11 files changed, 375 insertions(+), 32 deletions(-) Fusion-Task-Id: FN-7859 Fusion-Task-Lineage: 09b2035d-e8a0-438f-b1ab-1b0048b35c76 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
e559b2b538 |
FN-7853: preserve chat thread during active streaming turns
Fix useChat so already-rendered user/assistant messages no longer flicker away while an agent turn is actively streaming. - useChat.ts: during an active streaming turn for the current session, treat stale/empty/cross-session loadMessages responses as append-only against the visible thread instead of replacing it, merging any genuinely new same-session messages in and skipping the session-cache write when the active thread is being preserved. - ChatView.streaming-thread.test.tsx: add coverage asserting the rendered thread stays visible across mid-turn session-update/tool-call/stale-reload churn. - useChat.test.ts: add hook-level regression tests for the append-only/merge/cache-skip behavior during active streaming. - docs/architecture.md, docs/dashboard-guide.md: document the append-only mid-turn thread-stability behavior. - Add changeset (patch) for @runfusion/fusion describing the user-facing fix. Files changed: .../fn-7853-chat-mid-turn-message-stability.md | 7 + docs/architecture.md | 1 + docs/dashboard-guide.md | 1 + .../__tests__/ChatView.streaming-thread.test.tsx | 130 +++++++++++++ .../dashboard/app/hooks/__tests__/useChat.test.ts | 208 +++++++++++++++++++++ packages/dashboard/app/hooks/useChat.ts | 35 +++- 6 files changed, 380 insertions(+), 2 deletions(-) Fusion-Task-Id: FN-7853 Fusion-Task-Lineage: d9909469-082c-4eeb-81fb-b36d1a9e4705 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
|
c9d0211bec |
FN-7844: coordinate heartbeat and self-healing durable-agent error recovery
Unifies the two independent durable-agent error-recovery paths (heartbeat timer and self-healing sweep) so they share one retry budget, eligibility check, and audit surface instead of racing separate counters. - Share the heartbeatErrorRecovery attempt budget between HeartbeatMonitor's timer-entry recovery and SelfHealingManager.recoverOrphanedAgents(), with self-healing's legacy durableErrorRecovery metadata folded into the same counter via readHeartbeatErrorRetryCount(). - Add isHeartbeatErrorRecoverable() as the single transient/non-operator-actionable eligibility check, used by both the heartbeat timer and self-healing paths (self-healing additionally allows stale-worktree module-resolution errors). - resetHeartbeatErrorRecoveryMetadata() now strips the legacy durableErrorRecovery field so recovered agents don't retain stale sweep bookkeeping. - Self-healing emits the shared agent:auto-recover-error-state / agent:error-retry-exhausted run-audit events with source:"self-healing", and parks the agent paused with pauseReason:"error-retry-exhausted" on budget exhaustion, matching the heartbeat-timer behavior. - Update AGENTS.md, docs/architecture.md, and docs/agents.md to describe the consolidated recovery budget and audit surface. - Add a patch changeset documenting the fix for @runfusion/fusion. Files changed: .changeset/fn-7844-error-recovery-coordination.md | 7 ++ AGENTS.md | 2 +- docs/agents.md | 14 ++-- docs/architecture.md | 2 +- packages/engine/src/__tests__/heartbeat-error-recovery.test.ts | 13 +++- packages/engine/src/__tests__/self-healing.test.ts | 58 ++++++++++++++- packages/engine/src/agent-heartbeat.ts | 35 ++++++--- packages/engine/src/self-healing.ts | 85 ++++++++++++++++++---- 8 files changed, 180 insertions(+), 36 deletions(-) Fusion-Task-Id: FN-7844 Fusion-Task-Lineage: b70dcba5-56b6-412c-8be2-ef827bee9964 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |