diff --git a/docs/solutions/architecture-patterns/workflow-native-runtime-primitives.md b/docs/solutions/architecture-patterns/workflow-native-runtime-primitives.md new file mode 100644 index 0000000000..27d5458cfb --- /dev/null +++ b/docs/solutions/architecture-patterns/workflow-native-runtime-primitives.md @@ -0,0 +1,142 @@ +--- +title: "Workflow-native execution through runtime primitives" +date: 2026-06-09 +category: architecture-patterns +module: engine +problem_type: architecture_pattern +component: tooling +severity: high +applies_when: + - "A workflow graph claims lifecycle authority but nodes still call back into legacy orchestration" + - "Built-in workflows need to express default engine behavior without hidden executor branches" + - "Recovery, review, merge, or planning policy is split between workflow IR and imperative runtime code" + - "A graph runtime needs side effects without giving nodes direct access to scheduler or executor internals" +tags: [workflow-runtime, runtime-primitives, builtin-workflows, graph-executor, recovery-policy, engine-boundary] +related: + - docs/plans/2026-06-09-001-refactor-big-bang-workflow-native-execution-plan.md + - docs/solutions/architecture-patterns/per-entity-execution-principal-override-blast-radius.md +--- + +# Workflow-native execution through runtime primitives + +## Context + +The workflow graph executor already modeled task sequencing, but default task execution still depended on `WorkflowLegacySeams` that called back into `TaskExecutor.execute()`, review handoff, merge queue requests, and step-review helpers. That left two control planes: workflow IR described the lifecycle, while imperative code in the engine still decided large pieces of planning, execution, review, merge, and recovery. + +The cutover in PR #1536 introduced a typed runtime primitive boundary and primitive-backed node handlers. The engine still owns substrate concerns, but built-in workflows and node handlers now own lifecycle policy. + +Session history search: no relevant prior sessions were found for this specific workflow-runtime primitive cutover topic. + +## Guidance + +Use a **workflow policy / runtime primitive / engine substrate** split when moving lifecycle behavior into workflows. + +**Workflow policy** lives in IR and node routing: + +- the selected or built-in workflow defines node order and recovery branches; +- built-in lifecycle nodes express default behavior such as planning, execute, review, merge, parse-steps, step-review, and PR lifecycle; +- edge outcomes carry decisions like success, failure, revise, rethink, unavailable, manual-required, merge-timeout, or recovery routing. + +**Runtime primitives** are the only side-effect boundary nodes call: + +```ts +export interface WorkflowRuntimePrimitives { + prepareWorktree(ctx, task): Promise>; + runPlanningSession(ctx, task): Promise>; + runCodingSession(ctx, task, prepared): Promise>; + runTaskStep(ctx, task, stepIndex): Promise; + resetTaskStep(ctx, task, stepIndex, baselineSha?, checkpointId?): Promise; + runReview(ctx, task, input): Promise>; + requestMerge(ctx, task, input?): Promise>; + transitionTask(ctx, task, input): Promise; + abortRun(ctx, task, input): Promise; + audit(ctx, input): Promise | void; +} +``` + +Every primitive receives a workflow run context and node context, so audit, side-effect boundaries, effective principal, retry attempt, and recovery event identity do not have to be inferred from the mutable task row. + +**Engine substrate** remains centralized: + +- scheduler dispatch and routing claims; +- local vs remote node routing; +- persistence adapters and workflow run state; +- concurrency/semaphore control; +- process supervision and abort mechanics; +- storage access and audit/log sinks; +- non-bypassable guards such as file scope, worktree ownership, branch-group merge targets, and transition validation. + +In code, the graph executor should prefer primitive handlers when primitives are supplied: + +```ts +const promptLike = deps?.primitives + ? createPrimitivePromptLikeHandler(deps.primitives, runCustomNode) + : createPromptLikeHandler(seams, runCustomNode); +``` + +Keep legacy seam adapters only as a compatibility wrapper for older tests or transitional callers. Do not let the production workflow path re-enter a monolithic executor lifecycle after a workflow run starts. + +## Why This Matters + +The primitive boundary prevents graph nodes from becoming thin aliases for old orchestration. A workflow can decide *what happens next* while the engine still provides safe, centralized mechanics for *how a side effect happens*. + +This avoids several failure modes: + +- a graph node starts side effects, throws, and then the old executor reruns the same lifecycle; +- built-in workflows drift from hidden imperative branches; +- recovery sweeps mutate task lifecycle directly instead of waking/routing workflow recovery; +- effective principal, audit, or side-effect identity is reconstructed differently in each subsystem; +- tests prove graph traversal but miss the actual task lifecycle path. + +The cutover also exposed a useful testing boundary: old minimal executor-core fakes did not implement workflow-selection APIs. Those fakes should stay on the legacy characterization path unless the graph flag is explicitly enabled, while real workflow-aware stores can default unselected tasks to `builtin:coding`. + +## When to Apply + +- Converting an imperative task lifecycle into a graph/workflow runtime. +- Adding a built-in workflow that is supposed to be the compatibility contract for existing behavior. +- Moving recovery out of sweeps or background repair code and into routable workflow behavior. +- Introducing a side-effecting node kind that needs git, agents, review, merge, task transitions, or audit. +- Reviewing graph runtime changes where a node still calls a large executor method instead of a named primitive. + +## Examples + +### Built-in workflow owns the compatibility path + +Before the cutover, `builtin:coding` started at execute: + +```text +start -> execute -> review -> merge -> end +``` + +After the cutover, planning is explicit: + +```text +start -> planning -> execute -> review -> merge -> end +``` + +The workflow compiler also treats `planning` as a seam anchor before `execute`, so built-in graph shape and compile-time compatibility stay aligned. + +### Graph routing owns workflow-aware tasks first + +The production executor gives graph routing first claim for workflow-aware stores. It pins `workflowGraphExecutor` for the run, synthesizes `builtin:coding` for unselected tasks, and parks interpreter failures as workflow failures instead of falling through to legacy execution. + +Minimal old test fakes without workflow-selection support still fall through unless they explicitly enable the graph flag. This keeps characterization tests useful without weakening the production boundary. + +### Primitive-backed node handlers preserve routing semantics + +Primitive handlers map old seam names to explicit operations: + +- `planning` -> `runPlanningSession` +- `execute` -> `prepareWorktree` then `runCodingSession` +- `review` -> `runReview` +- `merge` -> `requestMerge` +- `step-execute` -> `runTaskStep` +- `step-review` -> `runReview` with step index and bounded unavailable retry + +The node still returns graph-native `{ outcome, value, contextPatch }`, so edge routing remains workflow-owned. + +## Related + +- `docs/workflow-steps.md` now documents the workflow runtime as authoritative and describes primitives as the side-effect boundary. +- `CONCEPTS.md` defines Workflow Runtime, Runtime Primitive, Built-in Lifecycle Node, and Recovery Event. +- `docs/solutions/architecture-patterns/per-entity-execution-principal-override-blast-radius.md` covers the adjacent identity/principal blast-radius checklist for workflow nodes.