Stacked on #2472 (`feature/workflow-vocabulary-b3-stranded-todo`). Test-only. No production file is touched. ## Why Every slice of this program has closed with the same caveat: *no renamed workflow was run against a live engine; all evidence is unit-level*. That caveat is load-bearing — eight times this session a test passed without exercising its subject. This PR removes it for the lifecycle spine. ## What actually runs `packages/engine/src/__tests__/workflow-lifecycle-live-e2e.pg.test.ts` drives the REAL pieces: - a **real PostgreSQL `TaskStore`** on a throwaway per-file database (shared PG harness; never the operator's DB, never port 4040), - the **real graph interpreter** (`WorkflowGraphTaskRunner`) with the **real column-boundary controller** wired to the **real `store.moveTask`** — all of its guards, traits, capacity reservation, and post-commit emission, - the **real scheduler release** (`runHoldReleaseSweep`), - the **real post-commit lifecycle bus** (`getWorkflowEventBus`), - the **real converted self-healing sweep** (`SelfHealingManager.recoverStrandedCompletedTodoTasks`, slice B3.1). Only the AI **seams** are scripted — the same boundary `testMode`/`mock` draws in production. **Assertion rule:** every lifecycle claim is asserted on **persisted state** (a fresh `getTask` with the store's task cache defeated, `run_audit_events` rows, `workflow_work_items` rows), never on "a function was called". The one spy — the event-bus subscriber — is asserted on the **received payload**, because the bus silently drops events that fail its shape check, so "emit was called" proves nothing. **Differential design:** the default-vocabulary (`todo`/`in-progress`/`in-review`/`done`) and renamed-vocabulary (`backlog`/`building`/`checking`/`shipped`) workflows come from ONE builder and differ ONLY in their four column ids. Any behavioral delta is attributable to the vocabulary alone. ## Coverage (9 tests, all green) | Scenario | What is proven | |---|---| | Default vocabulary, full spine | planning runs in the hold column, the card parks (graph does not self-promote), the **scheduler** performs hold→wip, the resumed run walks exec → review → merge-gate → end, persisted column is `done` | | **Renamed vocabulary, full spine** | identical, and no leg of the run touches any legacy column id | | Audit differential | the graph-owned boundary crossings are the same crossings node-for-node on both vocabularies; no legacy id appears in the renamed trail | | Event seam | a real subscriber **receives** a well-formed `TaskTransitioned` for the renamed `backlog`→`building` release and for the terminal move; `NodeEntered` arrives for every traversed node including `end` | | Crash / restart | exactly one durable continuation row at `exec`; a brand-new runner resumes from the row and the already-completed `planning` seam does **not** re-run; no duplicate continuation | | Converted sweep (B3.1) | a completed card in a **renamed** hold column is promoted (asserted on its persisted column), a card in the renamed **wip** column is not, and the default `todo` case still works | ## Mutation verification (both directions) Green suites are not evidence in this codebase, so both halves were falsified: 1. Keying `hold-release`'s `isHeldTask` on the `todo` literal → **5 of 6 spine tests fail, and the one that survives is the default-vocabulary one.** That is the exact signature the conversion program cares about. 2. Reverting slice B3.1's per-task hold-column resolution to the literal → **only the renamed stranded-todo test fails**; the default regression floor stays green. ## Findings surfaced by running it 1. **The IR validator refuses a `merge-blocker` column with no reachable merge-class node** ("the gate can never clear without one"). Kept rather than worked around — it means the review column here is genuinely gated. 2. **Entry into the merge region collapses to the legacy `merge` seam** (`MERGE_REGION_KINDS`), so a `merge-gate` node reaches the merge lane. Documented in the fixture. 3. **The transition policy refuses a direct hold → review move**, and it refuses it *workflow-resolved*: on the renamed board the only legal target is its own `building`, not `in-progress`. The recovery callback therefore promotes hold → wip → review rather than bypassing the policy. 4. **`moves.ts` still special-cases the `done` literal** (`if (toColumn === "done") clearNearDuplicateReferencesTo...`) after the post-commit emit. Not converted here and not in this PR's scope — flagged for the Phase B owner. ## Not driven end to end (stated plainly) - **Triage / specification.** The lifecycle starts from a task already bound to a workflow; `triage.ts` was not driven. The `planning` seam is scripted. - **Real merge.** No git worktree, no branch, no squash. `merge-gate` is pure policy; the `merge` seam is scripted. - **Lightweight / self-healing-off workflow.** The Tier 1 policy keys do not exist on this tip — there is no `policies` surface on the IR to set. Not drivable; not substituted with a unit test. - **Process-level crash.** The restart is an in-process one: a brand-new runner resuming from the persisted `workflow_work_items` row with no carried-over memory. No OS process was killed, so this proves durable-state resumption, not signal handling. ## Lane `.pg.test.ts` under the engine-default include glob, gated by `pgDescribe` so it skips cleanly with no PostgreSQL. The merge gate is untouched. Engine `tsc --noEmit` is clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added comprehensive live PostgreSQL workflow lifecycle coverage, including graph execution, suspension and resume, scheduler capacity release, crash recovery, and durable continuation. * Added validation for renamed workflow column configurations and columnless task movements. * Added event delivery checks for task transitions and node entry events. * Added self-healing recovery for stranded completed tasks in valid hold columns. * **Refactor** * Centralized workflow boundary handling, including task moves, continuation state, audit events, and diagnostics. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
120 lines
5.6 KiB
TypeScript
120 lines
5.6 KiB
TypeScript
/*
|
|
FNXC:WorkflowColumnBoundary 2026-07-27-16:40 (E2E validation seam — PR #2475 review, P2):
|
|
|
|
THE production wiring that binds the graph's column-boundary controller to the store, lifted out
|
|
of `Executor.buildColumnBoundaryHooks` so there is exactly ONE copy of it.
|
|
|
|
WHY IT MOVED. The E2E suite needs to drive the REAL boundary wiring, and `buildColumnBoundaryHooks`
|
|
was a private method reachable only by constructing a whole Executor. The test therefore rebuilt
|
|
the hooks by hand — and the hand copy DIVERGED in three ways within a single PR cycle (a broader
|
|
active-continuation filter, a missing graph-owned-move marker, and a different audit run id). A
|
|
test that rebuilds the wiring proves the copy works, which is the same failure shape as a spy that
|
|
passes on an event the bus refused. Extracting the factory is the smallest seam that lets the test
|
|
exercise the real thing; the Executor now delegates and owns only what is genuinely Executor state
|
|
(the in-flight move set and its logger), passed in as callbacks.
|
|
|
|
Behavior is byte-identical to the method it replaces — this is a move, not a rewrite.
|
|
*/
|
|
import type { Task, TaskStore, WorkflowWorkItemState } from "@fusion/core";
|
|
import { ACTIVE_WORKFLOW_WORK_ITEM_STATES } from "@fusion/core";
|
|
|
|
import { createStoreIrPinPersistence, type WorkflowIrPinStoreSurface } from "./workflow-column-boundary.js";
|
|
import type { WorkflowColumnBoundaryHooks } from "./workflow-graph-task-runner.js";
|
|
import { generateSyntheticRunId } from "./run-audit.js";
|
|
|
|
export interface ExecutorColumnBoundaryHooksDeps {
|
|
store: TaskStore;
|
|
task: Pick<Task, "id">;
|
|
/** The graph run id. Absent → the stable `${taskId}:workflow` fallback, as before. */
|
|
workflowRunId?: string;
|
|
/**
|
|
* Executor-owned side channel: a graph-owned lifecycle move is marked in-flight for its
|
|
* duration so the executor's own requeue path can tell "the graph moved this card" from "someone
|
|
* else moved this card" (executor.ts, `workflowLifecycleMovesInFlight` + `graphRouting`).
|
|
* Optional so a caller with no such state (the E2E harness) wires the rest of the real hooks
|
|
* without inventing an Executor.
|
|
*/
|
|
markMoveInFlight?: (taskId: string) => void;
|
|
clearMoveInFlight?: (taskId: string) => void;
|
|
/** Diagnostics sink for the boundary controller's warnings. */
|
|
onWarn?: (message: string, detail: Record<string, unknown>) => void;
|
|
}
|
|
|
|
/**
|
|
* Build the production column-boundary hooks: durable IR pin, capacity-suspension continuation,
|
|
* the graph-owned `moveTask`, and the ids-only run-audit emission.
|
|
*/
|
|
export function createExecutorColumnBoundaryHooks(
|
|
deps: ExecutorColumnBoundaryHooksDeps,
|
|
): WorkflowColumnBoundaryHooks {
|
|
const { store, task, workflowRunId } = deps;
|
|
// KTD-3 (U9b): store-backed durable IR pin. The cast is the same posture as
|
|
// buildBranchPersistence — structural probe of the row surface so a store
|
|
// lacking the pin fields degrades to the inert no-pin seam.
|
|
const pinPersistence = createStoreIrPinPersistence(
|
|
store as unknown as WorkflowIrPinStoreSurface,
|
|
task.id,
|
|
);
|
|
return {
|
|
pinNodeEntry: pinPersistence.pinNodeEntry,
|
|
loadPriorPin: pinPersistence.loadPriorPin,
|
|
// KTD-3 drift-park loop fix (PR #2342): detectDrift clears the stale pin
|
|
// row fields so an ordinary requeue re-resolves the CURRENT IR fresh.
|
|
clearPin: pinPersistence.clearPin,
|
|
onSuspend: async (suspension) => {
|
|
const items = await store.listWorkflowWorkItemsForTask(task.id, { kinds: ["task"] });
|
|
/* Only an ACTIVE row suppresses a fresh continuation. A cancelled/exhausted/manual-required
|
|
row is finished work, not a live wait — treating it as live would strand the card with no
|
|
continuation to resume from. */
|
|
const live = items.filter((item) =>
|
|
ACTIVE_WORKFLOW_WORK_ITEM_STATES.includes(item.state as WorkflowWorkItemState),
|
|
);
|
|
if (live.some((item) => item.nodeId === suspension.nodeId)) return;
|
|
await store.replaceActiveTaskWorkflowContinuation({
|
|
runId: `${workflowRunId ?? `${task.id}:workflow`}:continuation:${suspension.nodeId}:${items.length}`,
|
|
taskId: task.id,
|
|
nodeId: suspension.nodeId,
|
|
kind: "task",
|
|
state: "held",
|
|
stableWorkflowRunId: workflowRunId ?? `${task.id}:workflow`,
|
|
continuationSequence: items.length,
|
|
waitReason: "capacity",
|
|
sourceColumn: suspension.fromColumn,
|
|
targetColumn: suspension.toColumn,
|
|
irHash: suspension.irHash,
|
|
});
|
|
},
|
|
moveTask: async (toColumn, ctx) => {
|
|
deps.markMoveInFlight?.(task.id);
|
|
try {
|
|
await store.moveTask(task.id, toColumn, {
|
|
moveSource: "engine",
|
|
workflowMoveSource: "workflow-graph",
|
|
bypassGuards: true,
|
|
preserveProgress: true,
|
|
workflowMoveMetadata: { fromColumn: ctx.fromColumn, nodeId: ctx.nodeId },
|
|
});
|
|
} finally {
|
|
deps.clearMoveInFlight?.(task.id);
|
|
}
|
|
},
|
|
emitAudit: async (event) => {
|
|
await store.recordRunAuditEvent?.({
|
|
taskId: event.taskId,
|
|
agentId: "executor",
|
|
runId: generateSyntheticRunId("workflow-column-boundary", event.taskId),
|
|
domain: "database",
|
|
mutationType: event.type,
|
|
target: event.taskId,
|
|
metadata:
|
|
event.type === "task:column-transition"
|
|
? { taskId: event.taskId, workflowId: event.workflowId, fromColumn: event.fromColumn, toColumn: event.toColumn, nodeId: event.nodeId, irHash: event.irHash }
|
|
: { taskId: event.taskId, workflowId: event.workflowId, pinnedNodeId: event.pinnedNodeId, reason: event.reason },
|
|
});
|
|
},
|
|
onWarn: (message, detail) => {
|
|
deps.onWarn?.(message, detail);
|
|
},
|
|
};
|
|
}
|