Commit Graph

6559 Commits

Author SHA1 Message Date
Fusion (runfusion.ai)
a9dfb17b3d test(FN-4823): expand central-claim recovery coverage
Fusion-Task-Id: FN-4823
Fusion-Task-Lineage: 0034c04f-df82-4a1b-9f25-b93643a2c157
2026-05-16 21:26:04 -07:00
Fusion (runfusion.ai)
bbcc0a269c feat(FN-4823): wire central claim store into mesh lease manager
Fusion-Task-Id: FN-4823
Fusion-Task-Lineage: 0034c04f-df82-4a1b-9f25-b93643a2c157
2026-05-16 21:26:04 -07:00
Fusion (runfusion.ai)
6fa9aef630 feat(FN-4823): complete lease recovery central-claim reconciliation
Fusion-Task-Id: FN-4823
Fusion-Task-Lineage: 0034c04f-df82-4a1b-9f25-b93643a2c157
2026-05-16 21:26:04 -07:00
Fusion (runfusion.ai)
0b28388876 feat(FN-4804): complete Step 4 — add changeset
Fusion-Task-Id: FN-4804
Fusion-Task-Lineage: d2a98aa6-1c5c-427b-ba72-f5d229fdeeb9
2026-05-16 21:17:15 -07:00
Fusion (runfusion.ai)
9845678953 test(FN-4804): add quick chat dropdown regression coverage
Fusion-Task-Id: FN-4804
Fusion-Task-Lineage: d2a98aa6-1c5c-427b-ba72-f5d229fdeeb9
2026-05-16 21:17:15 -07:00
Fusion (runfusion.ai)
f5540a2a33 feat(FN-4804): complete Step 1 — preserve explicit session picks
Fusion-Task-Id: FN-4804
Fusion-Task-Lineage: d2a98aa6-1c5c-427b-ba72-f5d229fdeeb9
2026-05-16 21:17:15 -07:00
Fusion
5c36c0f15f fix(FN-4811): scope-leak guard always allows .changeset/ paths
The [scope-leak] reviewLevel=N enforcement=warn warning was firing on
many in-progress tasks for off-scope .changeset/FN-XXXX-*.md files (the
production signature on FN-4789, FN-4801, FN-4818 \u2014 their branches all
contained .changeset/FN-4811-*.md files from the in-progress fix stack).

By convention every task may add its own changeset entry under
.changeset/ per AGENTS.md 'Finalizing Changes' section, so .changeset/
files are now treated as always-allowed by the scope-leak guard
regardless of the task's declared file scope.

Cross-task changeset leakage is still caught by stronger downstream
guards (file-scope invariant at squash, post-merge audit) at much
higher signal-to-noise. This change only suppresses the noisy
per-execution warning that was flooding logs without adding any
defensive value.

Adds a new exported helper isAlwaysAllowedScopeLeakPath() so the
allowlist surface is easy to extend. Test coverage in
scope-leak-changeset-allowlist.test.ts.

Also (incorporated from interrupted merge state): loosens the
executing-task-lock.test.ts assertion that one losing-instance store
sees zero work-log entries rather than the brittle exact-count of
mockedCreateFnAgent invocations (the no-fn_task_done retry path can
fire on the winning instance, so the count varies).

Fusion-Task-Id: FN-4811
2026-05-16 21:15:10 -07:00
Fusion
b6df11a6a8 fix(FN-4811): process-wide executingTaskLock blocks parallel execute() across instances
Investigating FN-4814 + FN-4811 re-failures after commit 8bef30655 (which
added per-instance synchronous this.executing.add) revealed the per-instance
guard was insufficient. FN-4809 log at 02:48:17-18 UTC:

  02:48:17  [-]                Resuming execution after unpause
  02:48:17  [-]                Step 4 (Testing & Verification) -> pending
  02:48:17  [-]                Step 4 (Testing & Verification) -> pending
  02:48:17  [6097725-y2nb]     Executor detected stale merge state ...
  02:48:18  [6097816-9gde]     Executor detected stale merge state ...

Both runs y2nb and 9gde reached executor.ts:2661 (which is INSIDE execute(),
past the synchronous this.executing.add claim). The only viable explanation
is that there is more than one TaskExecutor instance in the same Node
process (engine restart race, multi-project hybrid runtime, or similar code
path). Each instance has its own executing Set, so the per-instance guard
doesn't help.

Fix: module-level singleton executingTaskLock in active-session-registry.ts,
shared across all TaskExecutor instances. execute() synchronously tryClaim()s
the lock; if false, bails. Every existing this.executing.delete() site also
calls executingTaskLock.release(). Per-instance this.executing kept because
many other call sites use it (this.executing.has at handler gates,
stuck-detector, resumeTaskForAgent, etc.).

Test setup (resetExecutorMocks in executor-test-helpers.ts) clears the lock
between tests so process-wide state doesn't leak (executor-pause and
executor-prompt tests would otherwise show 'expected 2 createFnAgent calls
but got 0' / 'expected not called but called 3 times' flakes).

Tests:
  - executing-task-lock.test.ts: 2 cases. Key case creates TWO TaskExecutor
    instances and races them on the same task ID, asserts only ONE actually
    runs. Verified FAILS on prior code (8bef30655) and PASSES on fix.

Verification:
  - Targeted suite (4 files, 170 tests): pass.
  - pnpm --filter @fusion/engine build: clean.
  - pnpm lint: clean.

Fusion-Task-Id: FN-4811
2026-05-16 21:06:51 -07:00
Fusion (runfusion.ai)
7ab3f7a750 feat(FN-4790): complete Step 5 — documentation and delivery
Fusion-Task-Id: FN-4790
Fusion-Task-Lineage: 4daf3560-7b02-4fbf-9e0a-45e048b795a1
2026-05-16 21:00:48 -07:00
Fusion (runfusion.ai)
30bf7df2bf test(FN-4790): complete Step 3 — add secrets crypto coverage
Fusion-Task-Id: FN-4790
Fusion-Task-Lineage: 4daf3560-7b02-4fbf-9e0a-45e048b795a1
2026-05-16 21:00:48 -07:00
Fusion (runfusion.ai)
5475c76e7e feat(FN-4790): complete Step 2 — export secrets crypto API
Fusion-Task-Id: FN-4790
Fusion-Task-Lineage: 4daf3560-7b02-4fbf-9e0a-45e048b795a1
2026-05-16 21:00:48 -07:00
Fusion (runfusion.ai)
63abf5cac4 feat(FN-4790): complete Step 1 — implement secrets crypto primitive
Fusion-Task-Id: FN-4790
Fusion-Task-Lineage: 4daf3560-7b02-4fbf-9e0a-45e048b795a1
2026-05-16 21:00:48 -07:00
Fusion (runfusion.ai)
27dd927213 feat(FN-4830): complete Steps 4-7 stale lock recovery delivery
Fusion-Task-Id: FN-4830
Fusion-Task-Lineage: d9b8ad72-669f-488e-8e85-be2dce9b8341
2026-05-16 20:54:39 -07:00
Fusion (runfusion.ai)
161cb565e6 feat(FN-4830): complete Step 3 — executor stale-lock recovery
Fusion-Task-Id: FN-4830
Fusion-Task-Lineage: d9b8ad72-669f-488e-8e85-be2dce9b8341
2026-05-16 20:54:34 -07:00
Fusion (runfusion.ai)
753068482e feat(FN-4830): complete Step 2 — backend stale-lock recovery
Fusion-Task-Id: FN-4830
Fusion-Task-Lineage: d9b8ad72-669f-488e-8e85-be2dce9b8341
2026-05-16 20:54:33 -07:00
Fusion (runfusion.ai)
9e2a2e37d0 feat(FN-4830): complete Step 1 — add stale lock helper
Fusion-Task-Id: FN-4830
Fusion-Task-Lineage: d9b8ad72-669f-488e-8e85-be2dce9b8341
2026-05-16 20:54:33 -07:00
Fusion (runfusion.ai)
a11bd0719e feat(FN-4822): complete Step 8 — document central claim authority
Fusion-Task-Id: FN-4822
Fusion-Task-Lineage: 08cc29e8-114a-48dc-80de-8d7fd2ce0e69
2026-05-16 20:48:00 -07:00
Fusion (runfusion.ai)
199f317813 feat(FN-4822): complete Step 6 — enforce central-claim race coverage
Fusion-Task-Id: FN-4822
Fusion-Task-Lineage: 08cc29e8-114a-48dc-80de-8d7fd2ce0e69
2026-05-16 20:48:00 -07:00
Fusion (runfusion.ai)
ad4b236985 test(FN-4822): complete Step 5 — add cross-node claim mutex integration race test
Fusion-Task-Id: FN-4822
Fusion-Task-Lineage: 08cc29e8-114a-48dc-80de-8d7fd2ce0e69
2026-05-16 20:48:00 -07:00
Fusion (runfusion.ai)
3190eac88b test(FN-4822): complete Step 4 — add AgentStore central-claim wiring tests
Fusion-Task-Id: FN-4822
Fusion-Task-Lineage: 08cc29e8-114a-48dc-80de-8d7fd2ce0e69
2026-05-16 20:48:00 -07:00
Fusion (runfusion.ai)
9d862d17a5 feat(FN-4822): complete Step 1 — add central task claim schema and API
Fusion-Task-Id: FN-4822
Fusion-Task-Lineage: 08cc29e8-114a-48dc-80de-8d7fd2ce0e69
2026-05-16 20:48:00 -07:00
Fusion (runfusion.ai)
bb9e93faba fix(FN-4818): align owning-node interaction assertions with landed behavior
Fusion-Task-Id: FN-4818
Fusion-Task-Lineage: 1619844a-3945-4681-af58-ca32e5533514
2026-05-16 20:27:40 -07:00
Fusion (runfusion.ai)
5c5b922d79 test(FN-4818): complete Step 2 — add owning-node handoff interaction coverage
Fusion-Task-Id: FN-4818
Fusion-Task-Lineage: 1619844a-3945-4681-af58-ca32e5533514
2026-05-16 20:27:40 -07:00
Fusion (runfusion.ai)
003b343b4e test(FN-4818): complete Step 1 — add claim mutex interaction coverage
Fusion-Task-Id: FN-4818
Fusion-Task-Lineage: 1619844a-3945-4681-af58-ca32e5533514
2026-05-16 20:27:40 -07:00
Fusion (runfusion.ai)
2617ae659f test(FN-4818): complete Step 1 — add multi-node claim mutex interaction coverage
Fusion-Task-Id: FN-4818
Fusion-Task-Lineage: 1619844a-3945-4681-af58-ca32e5533514
2026-05-16 20:27:39 -07:00
gsxdsm
4d5e296049 feat(FN-4801): merge fusion/fn-4801 2026-05-16 20:13:30 -07:00
Fusion (runfusion.ai)
9a249dfe2b fix(FN-4789): avoid bundling optional keytar native module
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:48 -07:00
Fusion (runfusion.ai)
937bed90b9 feat(FN-4789): complete Step 6 — changeset and resolver polish
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:48 -07:00
Fusion (runfusion.ai)
90cd39ebd2 feat(FN-4789): complete Step 2 — export manager and add optional keytar
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:48 -07:00
Fusion (runfusion.ai)
1a527fa753 fix(FN-4789): harden rotation semantics for active backend
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:47 -07:00
Fusion (runfusion.ai)
10e44eda3a feat(FN-4789): complete Step 1 — implement master key resolver
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:47 -07:00
Fusion (runfusion.ai)
e49c41185b test(FN-4789): add initial master key manager tests
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:47 -07:00
Fusion (runfusion.ai)
ff5f24f5e9 feat(FN-4789): complete Step 1 — implement master key manager
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:47 -07:00
Fusion (runfusion.ai)
338b1940f6 feat(FN-4810): complete Step 6 — finalize truncation docs and changeset
Fusion-Task-Id: FN-4810
Fusion-Task-Lineage: b7080f83-96b0-4106-9742-e14b20afeab1
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:47 -07:00
Fusion (runfusion.ai)
c3d1f7a95c feat(FN-4810): complete Step 3 — update scope-leak truncation docs
Fusion-Task-Id: FN-4810
Fusion-Task-Lineage: b7080f83-96b0-4106-9742-e14b20afeab1
Fusion-Task-Id: FN-4789
Fusion-Task-Lineage: cef0d12e-556e-43d8-9ce0-e4962c2d091b
2026-05-16 20:06:46 -07:00
Fusion (runfusion.ai)
1c7d49c5eb test(FN-4827): recover FN-4774 triage duplicate-detection regression block (orphan fusion/fn-4817; supersedes FN-4815) 2026-05-16 20:02:35 -07:00
Fusion (runfusion.ai)
8e3a635d3d feat(FN-4825): complete Step 4 — add changeset for node-unreachable audits
Fusion-Task-Id: FN-4825
Fusion-Task-Lineage: dc4e633e-fe46-47a8-9ecc-f032073caee9
2026-05-16 19:58:46 -07:00
Fusion (runfusion.ai)
14cf263bc6 feat(FN-4825): complete Step 3 — add scheduler unreachable-owner audits
Fusion-Task-Id: FN-4825
Fusion-Task-Lineage: dc4e633e-fe46-47a8-9ecc-f032073caee9
2026-05-16 19:58:44 -07:00
Fusion (runfusion.ai)
ee121a9f79 feat(FN-4825): complete Step 2 — emit mesh lease unreachable-owner audits
Fusion-Task-Id: FN-4825
Fusion-Task-Lineage: dc4e633e-fe46-47a8-9ecc-f032073caee9
2026-05-16 19:58:44 -07:00
Fusion (runfusion.ai)
2db5c10f07 feat(FN-4825): complete Step 1 — extend run-audit taxonomy
Fusion-Task-Id: FN-4825
Fusion-Task-Lineage: dc4e633e-fe46-47a8-9ecc-f032073caee9
2026-05-16 19:58:44 -07:00
Fusion (runfusion.ai)
cb9f4e4cd7 docs(FN-4814): complete Step 3 — wire readiness brief into docs
Fusion-Task-Id: FN-4814
Fusion-Task-Lineage: f73b5fce-6c97-4ba3-b7ae-81d85abf4834
2026-05-16 19:23:07 -07:00
Fusion (runfusion.ai)
2a8e3e5f0c fix(FN-4812): stabilize hoisted update-cache test fixture
Fusion-Task-Id: FN-4812
Fusion-Task-Lineage: 9877a784-5320-410c-82fa-3f6a6d9b5358
2026-05-16 19:16:48 -07:00
Fusion (runfusion.ai)
ec8d1f5246 test(FN-4812): assert mobile FileMentionPopup anchoring rule
Fusion-Task-Id: FN-4812
Fusion-Task-Lineage: 9877a784-5320-410c-82fa-3f6a6d9b5358
2026-05-16 19:16:48 -07:00
Fusion (runfusion.ai)
02713ee45e fix(FN-4812): anchor FileMentionPopup above input on mobile
Fusion-Task-Id: FN-4812
Fusion-Task-Lineage: 9877a784-5320-410c-82fa-3f6a6d9b5358
2026-05-16 19:16:47 -07:00
Fusion (runfusion.ai)
7cd5554641 feat(FN-4810): complete Step 6 — finalize truncation docs and changeset
Fusion-Task-Id: FN-4810
Fusion-Task-Lineage: b7080f83-96b0-4106-9742-e14b20afeab1
2026-05-16 18:57:58 -07:00
Fusion (runfusion.ai)
bd1c585b05 feat(FN-4810): complete Step 3 — update scope-leak truncation docs
Fusion-Task-Id: FN-4810
Fusion-Task-Lineage: b7080f83-96b0-4106-9742-e14b20afeab1
2026-05-16 18:57:58 -07:00
Fusion (runfusion.ai)
e68b448ce3 feat(FN-4810): complete Step 4 — document scope-leak truncation
Fusion-Task-Id: FN-4810
Fusion-Task-Lineage: b7080f83-96b0-4106-9742-e14b20afeab1
2026-05-16 18:57:58 -07:00
Fusion (runfusion.ai)
ea6501bed7 test(FN-4810): complete Step 2 — cover truncation behavior
Fusion-Task-Id: FN-4810
Fusion-Task-Lineage: b7080f83-96b0-4106-9742-e14b20afeab1
2026-05-16 18:57:58 -07:00
Fusion (runfusion.ai)
af4607ed54 feat(FN-4810): complete Step 1 — truncate scope-leak log rendering
Fusion-Task-Id: FN-4810
Fusion-Task-Lineage: b7080f83-96b0-4106-9742-e14b20afeab1
2026-05-16 18:57:58 -07:00
Fusion
8bef30655d fix(FN-4811): close concurrent-execute race that produced parallel runs
TaskExecutor.execute() had a classic JS async race window. Original:

  async execute(task) {
    if (this.executing.has(task.id)) return;                       // check
    const assignedAgentId = task.assignedAgentId;
    if (assignedAgentId && await this.shouldDeferForHeartbeat(...)) // AWAIT yields
      return;
    this.executing.add(task.id);                                   // add (too late)
    ...
  }

Two concurrent execute(task) calls (scheduler dispatch + task:moved event
handler + restart-recovery) both:
  1. Pass the synchronous has() check (Set is empty).
  2. Enter the awaited shouldDeferForHeartbeat call (yields the event loop).
  3. Resume and both call this.executing.add(task.id).
  4. Both proceed to create the same worktree path.

Production failure shape (FN-4814 + FN-4811, observed within minutes):

  01:30:56  [runA-caoe]  Worktree created at /...worktrees/bright-mesa
  01:30:56  [runB-w23q]  Worktree created at /...worktrees/bright-mesa
  01:30:58              worktree liveness assertion failed: not_usable_task_worktree
  01:31:48              [thirdRun] also fires liveness assertion fail
  01:37:48              In-review stall surfaced [no-worktree-no-merge-confirmed]

This is the root cause of the entire FN-4781/FN-4804/FN-4814/FN-4811
cascade. Every other guard added today (FN-4811 active-session gate,
self-healing reclaim defer, validation-failed recovery, silent reclaim
recovery, integrity-warning dedup) was patching SYMPTOMS of the
duplicate-run race. With this fix, the symptoms stop appearing.

Fix: claim the slot synchronously immediately after the has() check,
release it on the heartbeat-defer early-return path. No await happens
between check and claim, so the race window is closed.

Test added under
packages/engine/src/__tests__/reliability-interactions/concurrent-execute-race.test.ts
verified to fail on the prior (a1b1f9aa0) executor.ts and pass on the
fixed version:

  - Two concurrent execute() calls produce the SAME number of
    createFnAgent invocations as one execute() call (no amplification).
  - A second sequential execute() after the first completes IS allowed
    (slot was released).

The task must have assignedAgentId set to exercise the race \u2014 without
it, the short-circuit `assignedAgentId && ...` evaluates the left side
to false synchronously, and no await happens.

Full engine suite: 5048+ tests pass. The 7 transient test-file failures
in the broad parallel run are pre-existing flaky real-git tests
(branch-conflicts-zero-unique, branch-conflicts-recovery,
merger-overlap-guard subprocess-guard contention) \u2014 all of them pass
when run alone or as a smaller group, none touch the executor.execute()
path.

Fusion-Task-Id: FN-4811
2026-05-16 18:55:03 -07:00