feat(FN-5627): self-heal transient merge failures stuck at mergeRetries=3
After the FN-5627 merger fix (b2d547eae,230f6f45b) landed, two in-review tasks (FN-5628, FN-5632) remained stuck at mergeRetries=3 with status=failed because the merger correctly identified transient failure classes but had no auto-recovery path \u2014 the AUTO_MERGE_COOLDOWN_MS reset takes hours and gives up too easily. Failure classes covered: - lease-handoff-failed: target-not-queued (FN-5353/FN-5363 race where the merge queue lease was cleared between enqueue and handoff acquisition). - Legacy same-SHA spurious 'Integration branch X advanced concurrently (expected SHA, observed SHA)' errors from pre-FN-5627 code paths. Implementation: - New MergeDetails.transientRecoveryCount field tracks per-task recovery attempts, bounded by MAX_TRANSIENT_MERGE_RECOVERIES = 2. - New classifyTransientMergeError() string matcher in self-healing.ts identifies recoverable classes by error pattern. Returns null for genuine merge failures (verification, conflicts, real concurrent advances with different SHAs). - SelfHealingManager.recoverTransientMergeFailures() sweep finds matching in-review tasks, resets mergeRetries=0, clears status/error, increments recovery count, re-enqueues via requeueForAutoMerge. - Wired into BOTH startup recovery and periodic Batch 2 maintenance loop. - Emits merger:transient-failure-auto-recovered (recovered) and merger:transient-failure-budget-exhausted (terminal) audit events. No-op when autoMerge=false, requeueForAutoMerge not wired, or pause active. Repeat-suppression on budget-exhausted emit via error marker [transient-recovery-budget-exhausted] to prevent log spam. Tests (6 new): - target-not-queued recovery path - spurious-concurrent-advance-same-sha recovery path (legacy) - genuine concurrent-advance (different SHAs) NOT recovered - non-transient failures NOT recovered (verification, conflicts) - budget exhaustion emits marker once, no further requeue - autoMerge=false no-op Engine suite: 6157 tests pass (6 new). In-flight: FN-5628 and FN-5632 were manually reset via SQL so the already-running engine (which has the FN-5627 merger fix) can re-attempt their merges before this self-healing path lands and reloads. Future occurrences self-recover. Fusion-Task-Id: FN-5627
This commit is contained in:
@@ -1389,6 +1389,16 @@ export interface MergeDetails {
|
||||
warnedAt: string;
|
||||
reason: string;
|
||||
};
|
||||
/**
|
||||
* FN-5627 follow-up: counts how many times self-healing
|
||||
* `recoverTransientMergeFailures` has reset this task's `mergeRetries` and
|
||||
* re-enqueued it after a transient merge failure (e.g., `target-not-queued`
|
||||
* lease handoff race, or a misclassified same-SHA spurious concurrent-advance
|
||||
* left over from pre-FN-5627 code paths). Bounded by `MAX_TRANSIENT_MERGE_RECOVERIES`
|
||||
* (2) to avoid infinite recovery loops on genuinely-stuck tasks. Distinct from
|
||||
* `task.mergeRetries`, which counts in-cycle aiMergeTask retries.
|
||||
*/
|
||||
transientRecoveryCount?: number;
|
||||
}
|
||||
|
||||
/** Represents an agent's checkout lease on a task. */
|
||||
|
||||
Reference in New Issue
Block a user