FN-9168: Record terminal merge-boundary parks in run audit
Add failure-isolated audit visibility for merge-boundary proof failures without changing terminal park behavior. - Emit a redacted event from retry-boundary and graph-terminal park paths. - Bound audit sink latency and isolate absent, throwing, rejecting, or hung sinks. - Add reason-code coverage, catalogue documentation, and a patch changeset. Files changed: .changeset/fn-9168-merge-boundary-audit.md | 7 ++ AGENTS.md | 1 + docs/run-audit.md | 1 + .../src/__tests__/executor-graph-boundary.test.ts | 33 ++++++--- .../__tests__/merge-boundary-unproven-park.test.ts | 78 +++++++++++++++++++- .../executor/emit-merge-boundary-unproven-audit.ts | 82 ++++++++++++++++++++++ .../engine/src/executor/handle-graph-failure.ts | 23 +++++- .../executor/route-graph-merge-failure-to-retry.ts | 36 +++++++++- .../engine/src/executor/workflow-merge-boundary.ts | 38 ++++++++-- .../engine/src/run-audit/run-audit-catalogue.ts | 5 +- packages/engine/src/util/run-audit.ts | 10 +++ 11 files changed, 293 insertions(+), 21 deletions(-) Fusion-Task-Id: FN-9168 Fusion-Task-Lineage: cdeb5c1f-4b22-4531-878e-b18955e6ea5a Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
gsxdsm
7
.changeset/fn-9168-merge-boundary-audit.md
Normal file
7
.changeset/fn-9168-merge-boundary-audit.md
Normal file
@@ -0,0 +1,7 @@
|
||||
---
|
||||
"@runfusion/fusion": patch
|
||||
---
|
||||
|
||||
summary: Record merge-boundary proof parks in the run-audit history.
|
||||
category: feature
|
||||
dev: Adds `task:merge-boundary-unproven-parked` at both terminal park sites with closed reason codes and a bounded, failure-isolated emit seam.
|
||||
@@ -307,6 +307,7 @@ Scoped exception (FN-5819/FN-8823): while project auto-merge is On, shared-branc
|
||||
- Workspace (Phase D U1): self-healing emits `task:reconcile-workspace-partial-land` when it re-enqueues a partial/zero-landed workspace task's per-repo land (or parks it `failed` for proven branch absence or exhausted `evidence-unavailable` branch reads), and `task:reconcile-workspace-partial-land-no-action` when `autoMerge:false`, user-pause, a live sub-repo worktree (workspace-aware liveness), or `evidence-unavailable` blocks that backward move. The bounded evidence-exhaustion reason is `evidence-unavailable-exhausted`; audit metadata remains ids/counts/outcomes-only.
|
||||
- Workspace (Phase D U1): self-healing emits `task:reclaim-phantom-workspace-land-lease` when it clears a leaked `workspace-repo-land` lease whose owning task is terminal/dead and older than the FN-6736 staleness floor. Archived-role and soft-deleted owners are terminal; live merging, executing, or merge-pending owners are untouched.
|
||||
- FN-9164: `worktree:workspace-repo-base-branch` records per-repo base resolution with exactly `taskId`, `repoRelPath`, `stage`, `source`, `outcome`, and optional `fallbackReason`; branch/ref names are deliberately excluded from metadata and `target`, living only in the durable entry and task log.
|
||||
- FN-9168: `task:merge-boundary-unproven-parked` is emitted once per terminal merge-boundary-unproven park at the bounded-retry router and reachable graph terminal-merge park. Metadata is ids/counts/fixed outcomes only (`taskId`, `nodeId`, `failureValue`, `source`, optional `reasonCode`/`missingInstanceCount`, `priorColumn`, `priorStatus`, `outcome`) and never boundary reason prose, foreach instance IDs, or error text. `emitMergeBoundaryUnprovenParked` swallows absent/throwing/rejecting sinks and time-bounds a hung one with `MERGE_BOUNDARY_UNPROVEN_AUDIT_EMIT_TIMEOUT_MS`; late settlement is swallowed and the unref'd timer is cleared, so best-effort telemetry never alters, delays, aborts, or wedges the terminal park.
|
||||
- FN-9058: `worktree:workspace-main-checkout-edit` records workspace completion guard evidence with ids/counts/fixed outcomes only: task/repo IDs, file/commit counts, evidence or warning reason enum, `taskDoneRetryCount`, and `blocked`/`warned`/`skipped`; never paths, file content, or commit prose.
|
||||
- FN-9059: workspace coordination emits `workspace-lease:*` events for lease acquisition, renewal, release, `fence-published`, `fence-superseded`, `reclaimed`, and `reclaim-refused`, plus `workspace-land-intent:*` events for write-ahead intent lifecycle and `resolve-refused`. Metadata is ids, SHAs, counts, and fixed outcomes only; it never includes a credential-bearing remote URL.
|
||||
- FN-9056: self-healing emits `task:reconcile-orphaned-workspace-worktree` when it reclaims a complete-lane or conservatively-idle failed/soft-deleted workspace entry. It vetoes raw/canonical active paths, task-session/executor/merge liveness, pauses and scheduled recovery; archived rows remain archive-lifecycle-owned. It runs `git worktree prune` even for already-gone paths and deletes only safely-discardable canonical `fusion/<id>` branches. Duplicate, foreign, unowned, or outside-root claims are skipped without git work; one entry-scoped `MAX_STARVATION_DROPS` budget plus settlement bounds retries. Metadata is ids/counts/fixed outcomes: task/repo/path, success/reason/lane, worktree/prune/branch outcomes, and attempt.
|
||||
|
||||
@@ -35,6 +35,7 @@ Events that close a task's delivery: blocked/advanced completion parks, already-
|
||||
| `task:no-commits-finalize-blocked-incomplete-steps` | Finalize is blocked for a zero-commit task with incomplete workflow steps (FN-6461 lane). |
|
||||
| `task:empty-merge-finalize-blocked-no-landed-proof` | The AI empty-merge lane vetoes a zero-diff no-op finalize with no landed proof (FN-8141). |
|
||||
| `task:finalize-unproven-blocked` | Finalize is blocked because finalization has not been proven against the landing truth. |
|
||||
| `task:merge-boundary-unproven-parked` | A workflow merge boundary could not be proven and its terminal park is recorded with best-effort, time-bounded telemetry that never blocks or stalls the park. |
|
||||
| `task:finalize-lost-work-blocked` | Finalize is blocked because it would discard work (lost-work guard). |
|
||||
| `task:auto-recover-stale-merger-status` | Self-healing clears a stale merger status left on a finalize path. |
|
||||
|
||||
|
||||
@@ -222,19 +222,36 @@ describe("U5a — IR-driven merge boundary (scenario 1)", () => {
|
||||
expect(store.moveTask).toHaveBeenCalledWith("FN-B1", "in-review", expect.anything());
|
||||
});
|
||||
|
||||
it("keeps incomplete foreach coverage and zero expected steps blocked", async () => {
|
||||
for (const steps of [
|
||||
[{ id: "0", title: "Implement", status: "pending" as const }],
|
||||
[],
|
||||
]) {
|
||||
it("maps each incomplete merge-boundary proof to a redacted audit code", async () => {
|
||||
const cases = [
|
||||
{
|
||||
steps: [{ id: "0", title: "Implement", status: "pending" as const }],
|
||||
workflowStepResults: [],
|
||||
code: "no-node-result",
|
||||
missingInstanceCount: 1,
|
||||
},
|
||||
{
|
||||
steps: [{ id: "0", title: "Implement", status: "pending" as const }],
|
||||
workflowStepResults: [{ workflowStepId: "review", workflowStepName: "Review", source: "node" as const, phase: "pre-merge" as const, status: "pending" as const, completedAt: "2026-01-01" }],
|
||||
code: "non-terminal-node-result",
|
||||
missingInstanceCount: 1,
|
||||
},
|
||||
{
|
||||
steps: [{ id: "0", title: "Implement", status: "pending" as const }],
|
||||
workflowStepResults: [{ workflowStepId: "review", workflowStepName: "Review", source: "node" as const, phase: "pre-merge" as const, status: "passed" as const, completedAt: "2026-01-01" }],
|
||||
code: "missing-foreach-instances",
|
||||
missingInstanceCount: 1,
|
||||
},
|
||||
];
|
||||
for (const { steps, workflowStepResults, code, missingInstanceCount } of cases) {
|
||||
const { executor, liveTask } = makeExecutor({
|
||||
selection: { workflowId: "custom:foreach", stepIds: [] }, ir: foreachIr(), steps, workflowStepResults: [],
|
||||
selection: { workflowId: "custom:foreach", stepIds: [] }, ir: foreachIr(), steps, workflowStepResults,
|
||||
});
|
||||
const result = await executor.ensureWorkflowMergeBoundaryTask(
|
||||
liveTask,
|
||||
{ reason: "workflow-merge-boundary", nodeId: "merge", workflowId: "custom:foreach", runId: "r1" },
|
||||
) as { blocked?: { reason: string } };
|
||||
expect(result.blocked?.reason).toBe("no pre-merge node result recorded");
|
||||
) as { blocked?: { code: string; missingInstanceCount: number } };
|
||||
expect(result.blocked).toMatchObject({ code, missingInstanceCount });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -3,6 +3,7 @@ import { MERGE_BOUNDARY_UNPROVEN_VALUE, classifyMergePrimitiveResult, runWorkflo
|
||||
import { graphFailureValue, isMergeGraphFailure } from "../executor/graph-failure-pure.js";
|
||||
import { isTerminalMergeGraphFailureValue } from "../executor/task-predicates.js";
|
||||
import { routeGraphMergeFailureToRetry } from "../executor/route-graph-merge-failure-to-retry.js";
|
||||
import { MERGE_BOUNDARY_UNPROVEN_AUDIT_EMIT_TIMEOUT_MS } from "../executor/emit-merge-boundary-unproven-audit.js";
|
||||
import { shouldHoldActiveFileScopeLease } from "../scheduler.js";
|
||||
|
||||
const task = { id: "FN-9157", column: "in-review", steps: [], dependencies: [], log: [], createdAt: "2026-08-20T00:00:00.000Z", updatedAt: "2026-08-20T00:00:00.000Z", title: "t", description: "", prompt: "# t" } as any;
|
||||
@@ -36,7 +37,7 @@ describe("FN-9157 merge-boundary-unproven terminal routing", () => {
|
||||
store: { updateTask, logEntry } as any,
|
||||
getRunContextFor: () => undefined,
|
||||
mergeRequester,
|
||||
ensureWorkflowMergeBoundaryTask: vi.fn().mockResolvedValue({ task: live, blocked: { reason: "no pre-merge node result recorded" } }),
|
||||
ensureWorkflowMergeBoundaryTask: vi.fn().mockResolvedValue({ task: live, blocked: { reason: "no pre-merge node result recorded", code: "no-node-result", missingInstanceCount: 0 } }),
|
||||
persistTokenUsage: vi.fn(),
|
||||
}, live, graphResult(), undefined);
|
||||
expect(handled).toBe(true);
|
||||
@@ -45,4 +46,79 @@ describe("FN-9157 merge-boundary-unproven terminal routing", () => {
|
||||
expect(logEntry).toHaveBeenCalledWith("FN-9157", expect.stringContaining("retry parked task"), undefined, undefined);
|
||||
expect(shouldHoldActiveFileScopeLease({ ...live, status: "failed" }, [])).toBe(false);
|
||||
});
|
||||
|
||||
it("emits redacted audit metadata for parked and already-terminal retry boundaries", async () => {
|
||||
const live = { ...task, status: undefined };
|
||||
const updateTask = vi.fn(async (_id, patch) => ({ ...live, ...patch }));
|
||||
const recordRunAuditEvent = vi.fn().mockResolvedValue(undefined);
|
||||
const base = {
|
||||
store: { updateTask, logEntry: vi.fn(), recordRunAuditEvent } as any,
|
||||
getRunContextFor: () => undefined,
|
||||
mergeRequester: vi.fn(),
|
||||
persistTokenUsage: vi.fn(),
|
||||
};
|
||||
await expect(routeGraphMergeFailureToRetry({
|
||||
...base,
|
||||
ensureWorkflowMergeBoundaryTask: vi.fn().mockResolvedValue({ task: live, blocked: { reason: "foreach step instances incomplete at merge boundary: missing secret-a, secret-b", code: "missing-foreach-instances", missingInstanceCount: 2 } }),
|
||||
}, live, graphResult(), undefined)).resolves.toBe(true);
|
||||
expect(recordRunAuditEvent).toHaveBeenCalledTimes(1);
|
||||
expect(recordRunAuditEvent).toHaveBeenLastCalledWith(expect.objectContaining({
|
||||
mutationType: "task:merge-boundary-unproven-parked", target: "FN-9157",
|
||||
metadata: expect.objectContaining({ taskId: "FN-9157", source: "retry-boundary", reasonCode: "missing-foreach-instances", missingInstanceCount: 2, outcome: "parked" }),
|
||||
}));
|
||||
expect(JSON.stringify(recordRunAuditEvent.mock.calls[0][0].metadata)).not.toContain("secret-a");
|
||||
|
||||
const terminal = { ...live, status: "failed", error: "existing" };
|
||||
await expect(routeGraphMergeFailureToRetry({
|
||||
...base,
|
||||
ensureWorkflowMergeBoundaryTask: vi.fn().mockResolvedValue({ task: terminal, blocked: { reason: "no pre-merge node result recorded", code: "no-node-result", missingInstanceCount: 0 } }),
|
||||
}, terminal, graphResult(), undefined)).resolves.toBe(true);
|
||||
expect(recordRunAuditEvent.mock.calls[1][0].metadata.outcome).toBe("already-terminal");
|
||||
expect(updateTask).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["absent", undefined],
|
||||
["rejects", vi.fn().mockRejectedValue(new Error("audit sink down"))],
|
||||
["throws", vi.fn(() => { throw new Error("audit sink boom"); })],
|
||||
])("keeps the terminal park intact when the audit sink %s", async (_name, recordRunAuditEvent) => {
|
||||
const live = { ...task, status: undefined };
|
||||
const updateTask = vi.fn(async (_id, patch) => ({ ...live, ...patch }));
|
||||
const persistTokenUsage = vi.fn();
|
||||
await expect(routeGraphMergeFailureToRetry({
|
||||
store: { updateTask, logEntry: vi.fn(), recordRunAuditEvent } as any,
|
||||
getRunContextFor: () => undefined,
|
||||
mergeRequester: vi.fn(),
|
||||
ensureWorkflowMergeBoundaryTask: vi.fn().mockResolvedValue({ task: live, blocked: { reason: "no pre-merge node result recorded", code: "no-node-result", missingInstanceCount: 0 } }),
|
||||
persistTokenUsage,
|
||||
}, live, graphResult(), undefined)).resolves.toBe(true);
|
||||
expect(updateTask).toHaveBeenCalledWith("FN-9157", expect.objectContaining({ status: "failed", error: expect.stringContaining("MERGE_BOUNDARY_UNPROVEN:") }), undefined);
|
||||
expect(persistTokenUsage).toHaveBeenCalledWith("FN-9157");
|
||||
});
|
||||
|
||||
it("bounds a hung audit sink without skipping token usage", async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const live = { ...task, status: undefined };
|
||||
const updateTask = vi.fn(async (_id, patch) => ({ ...live, ...patch }));
|
||||
const persistTokenUsage = vi.fn();
|
||||
const handled = routeGraphMergeFailureToRetry({
|
||||
store: { updateTask, logEntry: vi.fn(), recordRunAuditEvent: vi.fn(() => new Promise<void>(() => {})) } as any,
|
||||
getRunContextFor: () => undefined,
|
||||
mergeRequester: vi.fn(),
|
||||
ensureWorkflowMergeBoundaryTask: vi.fn().mockResolvedValue({ task: live, blocked: { reason: "no pre-merge node result recorded", code: "no-node-result", missingInstanceCount: 0 } }),
|
||||
persistTokenUsage,
|
||||
}, live, graphResult(), undefined);
|
||||
let settled = false;
|
||||
void handled.then(() => { settled = true; });
|
||||
await Promise.resolve();
|
||||
expect(settled).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(MERGE_BOUNDARY_UNPROVEN_AUDIT_EMIT_TIMEOUT_MS);
|
||||
await expect(handled).resolves.toBe(true);
|
||||
expect(persistTokenUsage).toHaveBeenCalledWith("FN-9157");
|
||||
expect(updateTask).toHaveBeenCalledWith("FN-9157", expect.objectContaining({ error: expect.stringContaining("MERGE_BOUNDARY_UNPROVEN:") }), undefined);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
import type { TaskStore } from "@fusion/core";
|
||||
import { executorLog } from "../logger.js";
|
||||
import { generateSyntheticRunId } from "../util/run-audit.js";
|
||||
import type { MergeBoundaryUnprovenReasonCode } from "./workflow-merge-boundary.js";
|
||||
|
||||
export const MERGE_BOUNDARY_UNPROVEN_AUDIT_EMIT_TIMEOUT_MS = 2_000;
|
||||
|
||||
type MergeBoundaryUnprovenParkedAuditPayload = {
|
||||
taskId: string;
|
||||
nodeId: string;
|
||||
failureValue: string;
|
||||
source: "retry-boundary" | "graph-terminal-park";
|
||||
reasonCode?: MergeBoundaryUnprovenReasonCode;
|
||||
missingInstanceCount?: number;
|
||||
priorColumn: string;
|
||||
priorStatus: string | null | undefined;
|
||||
outcome: "parked" | "already-terminal";
|
||||
runId?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* FNXC:RunAudit 2026-08-20-02:00:
|
||||
* FN-9168 requires observability never to regress or stall delivery. The merge-boundary park is
|
||||
* terminal and correct on its own, so absent, throwing, rejecting, or never-settling audit sinks
|
||||
* are swallowed and time-bounded here. Callers await only ordering, never success: an unbounded
|
||||
* await after the terminal write would wedge the executor branch, skipping token persistence and
|
||||
* its return. Failure isolation is swallow-log-and-bound, with no retry, backoff, or queueing.
|
||||
*/
|
||||
export async function emitMergeBoundaryUnprovenParked(
|
||||
store: TaskStore | null | undefined,
|
||||
payload: MergeBoundaryUnprovenParkedAuditPayload,
|
||||
): Promise<void> {
|
||||
const sink = store?.recordRunAuditEvent;
|
||||
if (typeof sink !== "function") return;
|
||||
|
||||
let sinkPromise: Promise<unknown>;
|
||||
try {
|
||||
sinkPromise = Promise.resolve(sink.call(store, {
|
||||
taskId: payload.taskId,
|
||||
agentId: "executor",
|
||||
runId: payload.runId ?? generateSyntheticRunId("merge-boundary-unproven-park", payload.taskId),
|
||||
domain: "database",
|
||||
mutationType: "task:merge-boundary-unproven-parked",
|
||||
target: payload.taskId,
|
||||
metadata: {
|
||||
taskId: payload.taskId,
|
||||
nodeId: payload.nodeId,
|
||||
failureValue: payload.failureValue,
|
||||
source: payload.source,
|
||||
...(payload.reasonCode === undefined ? {} : { reasonCode: payload.reasonCode }),
|
||||
...(payload.missingInstanceCount === undefined ? {} : { missingInstanceCount: payload.missingInstanceCount }),
|
||||
priorColumn: payload.priorColumn,
|
||||
priorStatus: payload.priorStatus ?? null,
|
||||
outcome: payload.outcome,
|
||||
},
|
||||
}));
|
||||
} catch {
|
||||
executorLog.warn("[run-audit] failed to record task:merge-boundary-unproven-parked");
|
||||
return;
|
||||
}
|
||||
|
||||
// Observe late rejection before the bounded wait returns so it cannot become unhandled.
|
||||
void sinkPromise.catch(() => undefined);
|
||||
await new Promise<void>((resolve) => {
|
||||
const timer = setTimeout(() => {
|
||||
executorLog.warn("[run-audit] timed out recording task:merge-boundary-unproven-parked");
|
||||
resolve();
|
||||
}, MERGE_BOUNDARY_UNPROVEN_AUDIT_EMIT_TIMEOUT_MS);
|
||||
timer.unref?.();
|
||||
void sinkPromise.then(
|
||||
() => {
|
||||
clearTimeout(timer);
|
||||
resolve();
|
||||
},
|
||||
() => {
|
||||
clearTimeout(timer);
|
||||
executorLog.warn("[run-audit] failed to record task:merge-boundary-unproven-parked");
|
||||
resolve();
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
@@ -30,6 +30,8 @@ import { getPromptPath } from "../execution/spec-staleness.js";
|
||||
import { moveTaskToReplanColumn, resolveReplanTargetColumn } from "../execution/replan-target.js";
|
||||
import { executorLog } from "../logger.js";
|
||||
import { generateSyntheticRunId, type EngineRunContext } from "../util/run-audit.js";
|
||||
import { MERGE_BOUNDARY_UNPROVEN_VALUE } from "../workflows/workflow-merge-nodes.js";
|
||||
import { emitMergeBoundaryUnprovenParked } from "./emit-merge-boundary-unproven-audit.js";
|
||||
import { PAUSE_ABORT_PARK_ERROR_MARKER, PAUSE_ABORT_PARK_OPERATOR_MARKER } from "../self-healing.js";
|
||||
import {
|
||||
graphFailureValue,
|
||||
@@ -932,9 +934,28 @@ export async function handleGraphFailure(
|
||||
const message = `Workflow graph terminal merge failure at node '${failedNode ?? "unknown"}' (${failureValue}) — operator action required`;
|
||||
executorLog.warn(`${task.id}: ${message}`);
|
||||
await deps.store.logEntry(task.id, message, undefined, deps.getRunContextFor(task.id));
|
||||
if (live.status == null && live.error == null) {
|
||||
const outcome = live.status == null && live.error == null ? "parked" as const : "already-terminal" as const;
|
||||
if (outcome === "parked") {
|
||||
await deps.store.updateTask(task.id, { error: message, status: "failed" }, deps.getRunContextFor(task.id));
|
||||
}
|
||||
if (failureValue === MERGE_BOUNDARY_UNPROVEN_VALUE) {
|
||||
/*
|
||||
FNXC:RunAudit 2026-08-20-02:00:
|
||||
FN-9168 records this reachable graph-terminal merge-boundary-unproven park exactly once.
|
||||
Other terminal merge failures remain unchanged. The bounded emitter contains audit failure
|
||||
and hangs after the status write, so observability cannot alter or wedge the park.
|
||||
*/
|
||||
await emitMergeBoundaryUnprovenParked(deps.store, {
|
||||
taskId: task.id,
|
||||
nodeId: failedNode ?? "unknown",
|
||||
failureValue,
|
||||
source: "graph-terminal-park",
|
||||
priorColumn: live.column,
|
||||
priorStatus: live.status,
|
||||
outcome,
|
||||
runId: deps.getRunContextFor(task.id)?.runId,
|
||||
});
|
||||
}
|
||||
await deps.persistTokenUsage(task.id);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -13,6 +13,8 @@ import { graphFailureValue } from "./graph-failure-pure.js";
|
||||
import type { EngineRunContext } from "../util/run-audit.js";
|
||||
import { executorLog } from "../logger.js";
|
||||
import { MERGE_BOUNDARY_UNPROVEN_VALUE } from "../workflows/workflow-merge-nodes.js";
|
||||
import { emitMergeBoundaryUnprovenParked } from "./emit-merge-boundary-unproven-audit.js";
|
||||
import type { MergeBoundaryUnprovenReasonCode } from "./workflow-merge-boundary.js";
|
||||
|
||||
export type RouteGraphMergeFailureToRetryDeps = {
|
||||
store: TaskStore;
|
||||
@@ -21,7 +23,14 @@ export type RouteGraphMergeFailureToRetryDeps = {
|
||||
ensureWorkflowMergeBoundaryTask: (
|
||||
live: TaskDetail,
|
||||
opts: { reason: string; nodeId: string; workflowId: string; runId: string },
|
||||
) => Promise<{ task: TaskDetail; blocked?: { reason: string } }>;
|
||||
) => Promise<{
|
||||
task: TaskDetail;
|
||||
blocked?: {
|
||||
reason: string;
|
||||
code: MergeBoundaryUnprovenReasonCode;
|
||||
missingInstanceCount: number;
|
||||
};
|
||||
}>;
|
||||
persistTokenUsage: (taskId: string) => Promise<void>;
|
||||
};
|
||||
|
||||
@@ -52,15 +61,36 @@ export async function routeGraphMergeFailureToRetry(
|
||||
work, rather than silently retaining an in-review blocker.
|
||||
*/
|
||||
if (mergeBoundary.blocked) {
|
||||
const reason = mergeBoundary.blocked.reason;
|
||||
const { reason, code, missingInstanceCount } = mergeBoundary.blocked;
|
||||
await deps.store.logEntry(live.id, `Workflow merge boundary retry parked task: ${reason}`, undefined, deps.getRunContextFor(live.id));
|
||||
if (mergeBoundary.task.status !== "failed" || !mergeBoundary.task.error) {
|
||||
const outcome = mergeBoundary.task.status !== "failed" || !mergeBoundary.task.error
|
||||
? "parked" as const
|
||||
: "already-terminal" as const;
|
||||
if (outcome === "parked") {
|
||||
await deps.store.updateTask(
|
||||
live.id,
|
||||
{ status: "failed", error: `${MERGE_BOUNDARY_UNPROVEN_VALUE.toUpperCase().replaceAll("-", "_")}: ${reason}` },
|
||||
deps.getRunContextFor(live.id),
|
||||
);
|
||||
}
|
||||
/*
|
||||
FNXC:RunAudit 2026-08-20-02:00:
|
||||
FN-9168 records exactly one terminal merge-boundary-unproven park here. The boundary
|
||||
helper's blocked return is not a park and remains silent; its bounded audit seam contains
|
||||
failure and hangs, so telemetry cannot delay or alter this terminal write or return path.
|
||||
*/
|
||||
await emitMergeBoundaryUnprovenParked(deps.store, {
|
||||
taskId: live.id,
|
||||
nodeId: failedNode,
|
||||
failureValue: MERGE_BOUNDARY_UNPROVEN_VALUE,
|
||||
source: "retry-boundary",
|
||||
reasonCode: code,
|
||||
missingInstanceCount,
|
||||
priorColumn: live.column,
|
||||
priorStatus: live.status,
|
||||
outcome,
|
||||
runId: deps.getRunContextFor(live.id)?.runId,
|
||||
});
|
||||
await deps.persistTokenUsage(live.id);
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -17,9 +17,18 @@ export type WorkflowMergeBoundaryProof = {
|
||||
missingInstanceIds: string[];
|
||||
};
|
||||
|
||||
export type MergeBoundaryUnprovenReasonCode =
|
||||
| "no-node-result"
|
||||
| "non-terminal-node-result"
|
||||
| "missing-foreach-instances";
|
||||
|
||||
export type WorkflowMergeBoundaryResult = {
|
||||
task: TaskDetail;
|
||||
blocked?: { reason: string };
|
||||
blocked?: {
|
||||
reason: string;
|
||||
code: MergeBoundaryUnprovenReasonCode;
|
||||
missingInstanceCount: number;
|
||||
};
|
||||
};
|
||||
|
||||
export type WorkflowMergeBoundaryDeps = {
|
||||
@@ -81,13 +90,28 @@ export async function ensureWorkflowMergeBoundaryTask(
|
||||
*/
|
||||
const mergeProof = await deps.evaluateWorkflowMergeBoundary(live, metadata.runId);
|
||||
if (mergeProof.hasForeachStepExecute && !mergeProof.complete) {
|
||||
const reason = !mergeProof.hasRelevantNodeResult
|
||||
? "no pre-merge node result recorded"
|
||||
const blocked = !mergeProof.hasRelevantNodeResult
|
||||
? { reason: "no pre-merge node result recorded", code: "no-node-result" as const }
|
||||
: !mergeProof.allResultsTerminal
|
||||
? `non-terminal pre-merge node result ${mergeProof.nonTerminalResult?.workflowStepId ?? "unknown"} (${mergeProof.nonTerminalResult?.status ?? "unknown"})`
|
||||
: `foreach step instances incomplete at merge boundary: missing ${mergeProof.missingInstanceIds.join(", ")}`;
|
||||
await deps.store.logEntry(live.id, `Workflow merge boundary blocked: ${reason}`, undefined, deps.getRunContextFor(live.id));
|
||||
return { task: live, blocked: { reason } };
|
||||
? {
|
||||
reason: `non-terminal pre-merge node result ${mergeProof.nonTerminalResult?.workflowStepId ?? "unknown"} (${mergeProof.nonTerminalResult?.status ?? "unknown"})`,
|
||||
code: "non-terminal-node-result" as const,
|
||||
}
|
||||
: {
|
||||
reason: `foreach step instances incomplete at merge boundary: missing ${mergeProof.missingInstanceIds.join(", ")}`,
|
||||
code: "missing-foreach-instances" as const,
|
||||
};
|
||||
/*
|
||||
FNXC:RunAudit 2026-08-20-02:00:
|
||||
Boundary reason prose can contain foreach instance IDs and node-result status text. Run-audit
|
||||
metadata must remain ids/counts/outcomes-only, so terminal parks receive this closed code and
|
||||
missing-instance count rather than this human-readable reason.
|
||||
*/
|
||||
await deps.store.logEntry(live.id, `Workflow merge boundary blocked: ${blocked.reason}`, undefined, deps.getRunContextFor(live.id));
|
||||
return {
|
||||
task: live,
|
||||
blocked: { ...blocked, missingInstanceCount: mergeProof.missingInstanceIds.length },
|
||||
};
|
||||
}
|
||||
|
||||
if (deps.shouldCompleteChecklistAtWorkflowMerge(live, mergeProof)) {
|
||||
|
||||
@@ -43,6 +43,7 @@ export const DELIVERY_PIPELINE_RUN_AUDIT_EVENTS_LITERALS = [
|
||||
"task:no-commits-finalize-blocked-incomplete-steps",
|
||||
"task:empty-merge-finalize-blocked-no-landed-proof",
|
||||
"task:finalize-unproven-blocked",
|
||||
"task:merge-boundary-unproven-parked",
|
||||
"task:finalize-lost-work-blocked",
|
||||
"task:auto-recover-stale-merger-status",
|
||||
|
||||
@@ -69,7 +70,7 @@ export const DELIVERY_PIPELINE_RUN_AUDIT_EVENTS_LITERALS = [
|
||||
] as const;
|
||||
|
||||
/**
|
||||
* The 32-event literal union of the curated catalogue. `as const` preserves the literal element
|
||||
* The 31-event literal union of the curated catalogue. `as const` preserves the literal element
|
||||
* tuples so the notes map can be keyed by exactly the catalogued events (rather than widening to the
|
||||
* full `DatabaseMutationType` union). The exported array below is still typed as
|
||||
* `Readonly<DatabaseMutationType[]>`, so member-validity remains compile-time-enforced: assigning
|
||||
@@ -109,6 +110,8 @@ export const DELIVERY_PIPELINE_RUN_AUDIT_EVENT_NOTES: Readonly<Record<DeliveryPi
|
||||
"The AI empty-merge lane vetoes a zero-diff no-op finalize with no landed proof (FN-8141).",
|
||||
"task:finalize-unproven-blocked":
|
||||
"Finalize is blocked because finalization has not been proven against the landing truth.",
|
||||
"task:merge-boundary-unproven-parked":
|
||||
"A terminal merge-boundary proof failure is parked with bounded best-effort audit telemetry.",
|
||||
"task:finalize-lost-work-blocked":
|
||||
"Finalize is blocked because it would discard work (lost-work guard).",
|
||||
"task:auto-recover-stale-merger-status":
|
||||
|
||||
@@ -846,6 +846,16 @@ export type DatabaseMutationType =
|
||||
| "task:in-review-stall-deadlock-disposed"
|
||||
| "task:in-review-stall-terminal-provider-error"
|
||||
| "task:finalize-unproven-blocked"
|
||||
/**
|
||||
* FNXC:RunAudit 2026-08-20-02:02:
|
||||
* Records one terminal park when a workflow merge boundary cannot be proven, at the retry
|
||||
* boundary or graph-terminal park. Metadata is { taskId, nodeId, failureValue, source,
|
||||
* reasonCode?, missingInstanceCount?, priorColumn, priorStatus, outcome }; it is strictly
|
||||
* ids/counts/outcomes-only and never includes reason prose, instance IDs, or error text.
|
||||
* This is best-effort telemetry: an absent, failed, or hung write must not alter, block, or
|
||||
* stall the terminal park.
|
||||
*/
|
||||
| "task:merge-boundary-unproven-parked"
|
||||
/**
|
||||
* FN-5490/FN-5517/FN-5526/FN-5540 lost-work guard: the merger or self-heal
|
||||
* sweep refused to finalize a task as no-op because its record claimed
|
||||
|
||||
Reference in New Issue
Block a user