feat(FN-3857): auto-recover merge deadlocks by recovering already merged ca
Adds auto-recovery for merge deadlocks when candidates have already been merged on main (FN-3857), including tests, documentation updates in AGENTS.md, and a patch changeset for the `@runfusion/fusion` package. Fusion-Task-Id: FN-3857
This commit is contained in:
5
.changeset/fn-3857-auto-recover-merge-deadlocks.md
Normal file
5
.changeset/fn-3857-auto-recover-merge-deadlocks.md
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
---
|
||||||
|
"@runfusion/fusion": patch
|
||||||
|
---
|
||||||
|
|
||||||
|
Add automatic recovery for board-level merge deadlocks by promoting retry-exhausted already-landed review tasks to done, clearing stale `blockedBy` references on todo tasks when blockers are terminal or deadlocked, and excluding paused in-review worktrees from scheduler overlap `activeScopes` so paused blockers cannot repeatedly re-stamp downstream tasks.
|
||||||
@@ -178,6 +178,10 @@ Port 4040 is the production dashboard port. A user's live dashboard session is t
|
|||||||
- Run `kill`, `kill -9`, `pkill`, or `killall` against processes on port 4040
|
- Run `kill`, `kill -9`, `pkill`, or `killall` against processes on port 4040
|
||||||
- Start a test server on port 4040 — always use `--port 0` for random free port
|
- Start a test server on port 4040 — always use `--port 0` for random free port
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
- Merge deadlock self-healing now has three layered defenses: `SelfHealingManager.recoverAlreadyMergedReviewTasks()` and `SelfHealingManager.clearStaleBlockedBy()` in `packages/engine/src/self-healing.ts`, plus the paused-aware in-review scope filter in `packages/engine/src/scheduler.ts` (`inReviewWithWorktree` excludes `paused` tasks). Together these auto-finalize already-landed retry-exhausted review tasks, clear stale downstream blockers, and prevent paused review cards from re-blocking overlap dispatch.
|
||||||
|
|
||||||
## Engine Process Rules
|
## Engine Process Rules
|
||||||
|
|
||||||
The engine (`packages/engine`) runs the executor, merger, scheduler, IPC host, and dashboard-facing activity loop on a single Node event loop. **Blocking that loop stalls every task concurrently in-flight.**
|
The engine (`packages/engine`) runs the executor, merger, scheduler, IPC host, and dashboard-facing activity loop on a single Node event loop. **Blocking that loop stalls every task concurrently in-flight.**
|
||||||
|
|||||||
@@ -2600,7 +2600,7 @@ describe("SelfHealingManager", () => {
|
|||||||
managerWithRecovery.stop();
|
managerWithRecovery.stop();
|
||||||
});
|
});
|
||||||
|
|
||||||
it("filters out non-candidates", async () => {
|
it("filters out non-candidates but still evaluates paused failed candidates", async () => {
|
||||||
const managerWithRecovery = new SelfHealingManager(store, {
|
const managerWithRecovery = new SelfHealingManager(store, {
|
||||||
rootDir: "/tmp/test-project",
|
rootDir: "/tmp/test-project",
|
||||||
getExecutingTaskIds: () => new Set(["FN-executing"]),
|
getExecutingTaskIds: () => new Set(["FN-executing"]),
|
||||||
@@ -2613,12 +2613,17 @@ describe("SelfHealingManager", () => {
|
|||||||
{ id: "FN-executing", column: "in-review", paused: false, status: "failed", mergeRetries: 3, mergeDetails: undefined, log: [] },
|
{ id: "FN-executing", column: "in-review", paused: false, status: "failed", mergeRetries: 3, mergeDetails: undefined, log: [] },
|
||||||
{ id: "FN-confirmed", column: "in-review", paused: false, status: "failed", mergeRetries: 3, mergeDetails: { mergeConfirmed: true }, log: [] },
|
{ id: "FN-confirmed", column: "in-review", paused: false, status: "failed", mergeRetries: 3, mergeDetails: { mergeConfirmed: true }, log: [] },
|
||||||
]);
|
]);
|
||||||
|
mockedExecSync.mockImplementation(() => "" as any);
|
||||||
|
|
||||||
const result = await managerWithRecovery.recoverAlreadyMergedReviewTasks();
|
const result = await managerWithRecovery.recoverAlreadyMergedReviewTasks();
|
||||||
|
|
||||||
expect(result).toBe(0);
|
expect(result).toBe(0);
|
||||||
expect(store.updateTask).not.toHaveBeenCalled();
|
expect(store.updateTask).not.toHaveBeenCalled();
|
||||||
expect(store.moveTask).not.toHaveBeenCalled();
|
expect(store.moveTask).not.toHaveBeenCalled();
|
||||||
|
expect(mockedExecSync).toHaveBeenCalledWith(
|
||||||
|
expect.stringContaining("Fusion-Task-Id: FN-paused"),
|
||||||
|
expect.any(Object),
|
||||||
|
);
|
||||||
|
|
||||||
managerWithRecovery.stop();
|
managerWithRecovery.stop();
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1807,7 +1807,6 @@ export class SelfHealingManager {
|
|||||||
*
|
*
|
||||||
* Candidate filter:
|
* Candidate filter:
|
||||||
* - `column === "in-review"`
|
* - `column === "in-review"`
|
||||||
* - not paused
|
|
||||||
* - `status === "failed"`
|
* - `status === "failed"`
|
||||||
* - `(mergeRetries ?? 0) >= MAX_AUTO_MERGE_RETRIES`
|
* - `(mergeRetries ?? 0) >= MAX_AUTO_MERGE_RETRIES`
|
||||||
* - `mergeDetails.mergeConfirmed !== true`
|
* - `mergeDetails.mergeConfirmed !== true`
|
||||||
@@ -1830,7 +1829,6 @@ export class SelfHealingManager {
|
|||||||
const tasks = await this.store.listTasks({ column: "in-review", slim: true });
|
const tasks = await this.store.listTasks({ column: "in-review", slim: true });
|
||||||
const candidates = tasks.filter((task) =>
|
const candidates = tasks.filter((task) =>
|
||||||
task.column === "in-review" &&
|
task.column === "in-review" &&
|
||||||
!task.paused &&
|
|
||||||
task.status === "failed" &&
|
task.status === "failed" &&
|
||||||
(task.mergeRetries ?? 0) >= MAX_AUTO_MERGE_RETRIES &&
|
(task.mergeRetries ?? 0) >= MAX_AUTO_MERGE_RETRIES &&
|
||||||
task.mergeDetails?.mergeConfirmed !== true &&
|
task.mergeDetails?.mergeConfirmed !== true &&
|
||||||
|
|||||||
Reference in New Issue
Block a user