fix: preserve gridlock notification cooldown (#3469)
## Summary - preserve the gridlock notification wall-clock cooldown across transient detector clears - add a regression test for clear-then-rediscover behavior during the cooldown - document the cooldown contract and add a patch changeset ## Test plan - `corepack pnpm --filter @fusion/engine exec vitest run src/__tests__/notifier.test.ts --project=engine-default --reporter=verbose -t 'suppresses the same gridlock after a transient resolution during cooldown'` - `corepack pnpm --filter @fusion/engine typecheck` - `corepack pnpm build` - `corepack pnpm changeset status --output /tmp/fusion-gridlock-changeset-status.json` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Gridlock notifications now remain suppressed during the 15-minute cooldown, even if the condition temporarily clears and reappears. - Prevents repeated notifications caused by transient detector-state changes. - **Documentation** - Updated gridlock notification behavior documentation to reflect the persistent cooldown. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
7
.changeset/fix-gridlock-notification-cooldown.md
Normal file
7
.changeset/fix-gridlock-notification-cooldown.md
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
---
|
||||||
|
"@runfusion/fusion": patch
|
||||||
|
---
|
||||||
|
|
||||||
|
summary: Prevent repeated gridlock alerts when detection briefly clears.
|
||||||
|
category: fix
|
||||||
|
dev: Preserves the wall-clock ntfy cooldown across transient gridlock detector clears.
|
||||||
@@ -811,7 +811,7 @@ Guardrails: this routine does **not** retry merges, does **not** apply to mixed/
|
|||||||
- Runtime ownership: `NtfyNotifier` no longer owns an independent task-lifecycle listener graph; `ProjectEngine` injects the canonical `NotificationService` instance so task lifecycle notifications (`task:created`, `task:moved`, `task:updated`, `task:merged`) are emitted through a single path.
|
- Runtime ownership: `NtfyNotifier` no longer owns an independent task-lifecycle listener graph; `ProjectEngine` injects the canonical `NotificationService` instance so task lifecycle notifications (`task:created`, `task:moved`, `task:updated`, `task:merged`) are emitted through a single path.
|
||||||
- Merge dedup safety: all merge-success → done code paths (direct merger completion, owned/no-op auto-finalize, mergeConfirmed fast-path, PR-strategy finalize, and merge-success self-healing finalizers) emit `store.emit("task:merged", result)` with a merged `MergeResult`. `NotificationService.notifiedEvents` remains the single dedup source of truth, so duplicate upstream emits still produce exactly one canonical `merged` ntfy lifecycle notification per task.
|
- Merge dedup safety: all merge-success → done code paths (direct merger completion, owned/no-op auto-finalize, mergeConfirmed fast-path, PR-strategy finalize, and merge-success self-healing finalizers) emit `store.emit("task:merged", result)` with a merged `MergeResult`. `NotificationService.notifiedEvents` remains the single dedup source of truth, so duplicate upstream emits still produce exactly one canonical `merged` ntfy lifecycle notification per task.
|
||||||
- Compatibility scope: `NtfyNotifier` remains responsible for gridlock-only compatibility notifications (`notifyGridlock`) and legacy helper APIs.
|
- Compatibility scope: `NtfyNotifier` remains responsible for gridlock-only compatibility notifications (`notifyGridlock`) and legacy helper APIs.
|
||||||
- Legacy gridlock ntfy delivery is cooldown-throttled: first detection notifies immediately, subsequent detections are suppressed for 15 minutes (even if blocked-task membership changes), and the cooldown resets as soon as gridlock fully clears.
|
- Legacy gridlock ntfy delivery is cooldown-throttled: first detection notifies immediately, subsequent detections are suppressed for 15 minutes (even if blocked-task membership changes or detector state transiently clears), and a later detection may notify after the wall-clock cooldown expires.
|
||||||
- `NotificationService` (`notification/notification-service.ts`) — provider lifecycle + event dispatch orchestration
|
- `NotificationService` (`notification/notification-service.ts`) — provider lifecycle + event dispatch orchestration
|
||||||
- Subscribes to task lifecycle events plus mailbox and memory events. `task:created` dispatches `task-created` only when `task.sourceAgentId` is present (agent-created tasks, including fn task-create calls made by agents). `message:sent` dispatches `message:agent-to-user` and `message:agent-to-agent` notification events (with message metadata for deep-links), and manual `POST /api/memory/dream` processing emits `store.emit("memory:dreams-processed", payload)` when new DREAMS content is written.
|
- Subscribes to task lifecycle events plus mailbox and memory events. `task:created` dispatches `task-created` only when `task.sourceAgentId` is present (agent-created tasks, including fn task-create calls made by agents). `message:sent` dispatches `message:agent-to-user` and `message:agent-to-agent` notification events (with message metadata for deep-links), and manual `POST /api/memory/dream` processing emits `store.emit("memory:dreams-processed", payload)` when new DREAMS content is written.
|
||||||
- `failed` task notifications are deferred behind a grace window (default 60s) and suppressed when recovery signals arrive (`column=done`, `mergeDetails.mergeConfirmed=true`, or status clear with an `Auto-recovered:` log). Persistent failures still emit exactly once after the window.
|
- `failed` task notifications are deferred behind a grace window (default 60s) and suppressed when recovery signals arrive (`column=done`, `mergeDetails.mergeConfirmed=true`, or status clear with an `Auto-recovered:` log). Persistent failures still emit exactly once after the window.
|
||||||
|
|||||||
@@ -663,30 +663,28 @@ describe("NtfyNotifier", () => {
|
|||||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("allows a new gridlock notification immediately after resolution reset", async () => {
|
// FNXC:Notifications 2026-08-16-16:03: A transient clear must not re-arm the
|
||||||
|
// same gridlock notification while the wall-clock cooldown is still active.
|
||||||
|
it("suppresses the same gridlock after a transient resolution during cooldown", async () => {
|
||||||
store.setSettings({ ntfyEnabled: true, ntfyTopic: "test-topic", ntfyEvents: ["gridlock"] });
|
store.setSettings({ ntfyEnabled: true, ntfyTopic: "test-topic", ntfyEvents: ["gridlock"] });
|
||||||
fetchMock.mockResolvedValue({ ok: true });
|
fetchMock.mockResolvedValue({ ok: true });
|
||||||
notifier = new NtfyNotifier(store);
|
notifier = new NtfyNotifier(store);
|
||||||
await notifier.start();
|
await notifier.start();
|
||||||
|
|
||||||
notifier.notifyGridlock({
|
const event = {
|
||||||
blockedTaskCount: 1,
|
blockedTaskCount: 1,
|
||||||
reasons: { "FN-001": "dependency" },
|
reasons: { "FN-001": "dependency" as const },
|
||||||
blockedTaskIds: ["FN-001"],
|
blockedTaskIds: ["FN-001"],
|
||||||
blockingTaskIds: ["FN-002"],
|
blockingTaskIds: ["FN-002"],
|
||||||
});
|
};
|
||||||
|
notifier.notifyGridlock(event);
|
||||||
|
|
||||||
vi.advanceTimersByTime(60_000);
|
vi.advanceTimersByTime(60_000);
|
||||||
notifier.notifyGridlock(null);
|
notifier.notifyGridlock(null);
|
||||||
notifier.notifyGridlock({
|
notifier.notifyGridlock(event);
|
||||||
blockedTaskCount: 1,
|
|
||||||
reasons: { "FN-009": "overlap" },
|
|
||||||
blockedTaskIds: ["FN-009"],
|
|
||||||
blockingTaskIds: ["FN-010"],
|
|
||||||
});
|
|
||||||
|
|
||||||
await flushAsyncWork();
|
await flushAsyncWork();
|
||||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -493,7 +493,11 @@ export class NtfyNotifier {
|
|||||||
|
|
||||||
notifyGridlock(event: GridlockEvent | null): void {
|
notifyGridlock(event: GridlockEvent | null): void {
|
||||||
if (event === null) {
|
if (event === null) {
|
||||||
this.lastGridlockNotificationAt = null;
|
/*
|
||||||
|
FNXC:Notifications 2026-08-16-16:03:
|
||||||
|
Detector state may flap clear between scheduler polls while the same underlying gridlock persists.
|
||||||
|
Keep the wall-clock cooldown so a transient clear cannot re-arm ntfy every minute.
|
||||||
|
*/
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user