test(engine): column-agent surface matrix, parity invisibility proof, docs
U7: full mode × surface × own-settings matrix ledger with 5 gap-filling tests, default-workflow zero-binding parity assertions, changeset covering the complete feature, and a workflow-steps.md authoring section for column agents.
This commit is contained in:
gsxdsm
@@ -7,3 +7,5 @@ Add per-column agent assignment for workflow columns, behind the combined `exper
|
||||
A workflow column can now name a permanent agent from the registry plus a mode — `defer` (the column agent is the default for work in that column that carries no agent/model settings of its own) or `override` (the column agent supersedes node- and task-level agent/model settings). The binding applies to all session-running work attributable to the column's nodes: custom prompt/gate/script nodes, the execute seam's coding session, and step-execute sessions. Precedence is resolved by one shared `@fusion/core` resolver (`resolveColumnAgentBinding` + `resolveEffectiveAgent`) consumed by every reader, with defer/override expressed as explicit named rules and defer granularity all-or-nothing (an own agent identity OR a complete `modelProvider`+`modelId` pair suppresses the column agent). The binding keys off the node's declared IR column; foreach template nodes inherit the enclosing foreach node's column. A missing/deleted agent at resolution time logs and falls back to normal resolution — a live session is never aborted. The built-in default workflow carries no column agents and stays byte-identical (parity oracle); with either flag off, column agents are inert.
|
||||
|
||||
The effective column agent is also the principal for the subsystems that previously assumed the running agent is always `task.assignedAgentId`: action gating (`buildActionGateContext` / `buildPermanentAgentGatingContext`) is computed for the agent actually running; heartbeat serialization honors it in both directions (the execute deferral gate, a second `resumeTaskForAgent` pass that re-dispatches tasks whose effective column agent matches, and a reverse-direction heartbeat-scheduler guard so an `allowParallelExecution=false` column agent never heartbeats concurrently with its own session); and a workflow-definition edit or agent runtimeConfig change that re-keys the column-effective agent/model hot-swaps the running graph session, while an agent deleted mid-session falls back without a restart.
|
||||
|
||||
Authoring lands in the workflow editor: the column panel gains a registry-backed per-column agent picker plus a defer/override mode toggle, bound columns are badged on their headers, and a node inside an override column shows that its own executor settings are superseded (so override never reads as a bug). Picker interaction states are explicit — flags off disables the picker with a tooltip naming both required flags, an in-flight fetch disables it, a failed fetch shows an inline error, and a stored `agentId` missing from the registry renders an "Agent not found" warning that preserves the IR until the author clears or replaces it. Agent references are validated at save time: the `POST`/`PATCH` workflow routes reject an unknown `agentId` with a typed 4xx naming the offending column, and binding an agent whose permission policy is broader than the project default requires an explicit `confirmPolicyEscalation` flag so override cannot silently re-key action gates to a more-privileged agent.
|
||||
|
||||
@@ -61,6 +61,36 @@ FN-5769 evaluated whether those conventions required a `1.1.0` schema bump and r
|
||||
|
||||
The `workflowColumns` track introduces **IR v2** (`version: "v2"`), where a workflow additionally defines its own **columns** (`{ id, name, traits: [{ trait, config }] }`), places nodes in columns (`node.column`), and gains `hold`, `split`, and `join` node kinds. Columns become first-class, workflow-defined task state carrying composable **traits** (declarative flags + lifecycle hooks); this generalizes the fixed pipeline + the `gateMode` semantics documented below into per-column trait configuration. v1 graphs still parse and upgrade by synthesizing default-workflow columns. The column/trait model — the trait vocabulary, the substrate/policy line, the transition authority, and the graduation gate — is documented in **`docs/architecture.md` § 9 "Workflow-defined columns & traits"** and the **Concepts** glossary (column, trait, lane, hold node, split/join, default workflow, `transitionPending`). The whole v2 model is gated behind `experimentalFeatures.workflowColumns`; with the flag off, the v1 IR and the quality-gate `WorkflowStep` model below are unchanged.
|
||||
|
||||
### Workflow IR v2 — per-column agent assignment
|
||||
|
||||
A v2 column can optionally name a **permanent agent** from the agent registry, staffing every card that flows through it once instead of node-by-node or task-by-task. The binding is a first-class optional field on the column (not a trait — traits are board-transition policy; this is execution identity):
|
||||
|
||||
```
|
||||
{ id: "review", name: "Review", traits: [],
|
||||
agent: { agentId: "agent-001", mode: "defer" | "override" } }
|
||||
```
|
||||
|
||||
**Binding shape.** `agent.agentId` is a non-empty registry agent id; `agent.mode` is `defer` or `override`. The field is omitted entirely when unset — a column with no `agent` key yields no binding, and the built-in default workflow carries none (it stays byte-identical, the parity oracle). Adding a binding forces the workflow to v2.
|
||||
|
||||
**Which column governs.** The binding keys off the node's **declared** IR column (`node.column`), never the task's current board lane. A node with no declared column resolves normally (no column agent), even when other columns carry override bindings.
|
||||
|
||||
**`defer` vs `override`.**
|
||||
|
||||
- **`defer`** — the column agent is the default *only* when the work carries no agent/model settings of its own. "Own settings" is all-or-nothing: an own agent identity **or** a complete `modelProvider`+`modelId` pair suppresses the column agent entirely. An incomplete model pair (provider with no model id) does **not** count as own settings, so the column agent still wins (matching the executor's both-present model rule). The column agent is never blended with own settings — filling only the missing half would create hybrid identities that are impossible to audit.
|
||||
- **`override`** — the column agent supersedes node-level and task-level agent/model settings: identity, model, **and** persona.
|
||||
|
||||
**Where it applies.** The effective agent governs all session-running work attributable to the column's nodes: custom prompt/gate/script nodes, the execute seam's coding session, and step-execute sessions. Raw CLI script nodes run no session, so the binding is a no-op there (the skip is audited). Every adoption is logged (`running as column agent '<id>' (<mode>)`) so the audit trail explains who ran and why.
|
||||
|
||||
**Foreach template inheritance.** A node inside a `foreach` template subgraph inherits the **enclosing foreach node's** column, unless the template node declares its own `column` (which then wins). Each per-step instance session is attributed to the resolved column agent.
|
||||
|
||||
**Principal semantics.** The effective column agent becomes the **principal**, not merely a model source. Action gating is computed for the agent actually running (a security boundary — never `task.assignedAgentId` when an override governs). Heartbeat serialization follows it in both directions: a column agent with `allowParallelExecution=false` is serialized like an assigned agent, the engine re-dispatches tasks whose *effective* column agent matches (not only `assignedAgentId` matches), and the heartbeat scheduler never lets a column agent heartbeat concurrently with its own override session. A workflow-definition edit or agent `runtimeConfig` change that re-keys the effective agent/model hot-swaps a running session, the same way a `task.modelProvider` change does today.
|
||||
|
||||
**Missing-agent fallback.** A missing or deleted agent at resolution time logs and falls back to normal resolution — a live session is never aborted because its column agent was deleted mid-flight.
|
||||
|
||||
**Flag requirements.** Column agents act only when **both** `experimentalFeatures.workflowColumns` and `experimentalFeatures.workflowGraphExecutor` are on; with either off the binding is inert (config is still stored and round-trips — only execution is gated), and the editor surfaces that the picker is disabled with a tooltip naming both flags.
|
||||
|
||||
**Write-time validation.** Saving a workflow validates agent references: an unknown `agentId` is rejected with a typed 4xx naming the column. Binding an agent whose permission policy is broader than the project default requires an explicit policy-escalation confirmation (`confirmPolicyEscalation`) at save time, so override cannot silently re-key action gates to a more-privileged agent.
|
||||
|
||||
### Workflow IR v2 — step inversion (foreach, step-review, parse-steps, code)
|
||||
|
||||
The **step-inversion** track makes task *steps* themselves workflow-modelable. Today the engine owns step policy end-to-end (PROMPT.md parsing, per-step review verdicts, RETHINK/REVISE control flow, merge blocking). Step inversion extracts exactly one new substrate capability — *run one step inside a task's session, and reset one step to its baseline* — and exposes everything else as authored graph structure. It is additive to IR v2 and gated by `experimentalFeatures.workflowGraphExecutor`. The default coding workflow is untouched and byte-identical (it keeps its monolithic `execute` seam and is the parity oracle); inversion is opt-in via custom workflows and a new built-in **stepwise coding workflow**.
|
||||
|
||||
@@ -144,6 +144,26 @@ describe("runGraphCustomNode column-agent resolution (plan U3)", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("override column: bare node adopts the column agent (own-absent cell)", async () => {
|
||||
// override × own-absent: nothing to supersede, the column agent is adopted.
|
||||
const store = createMockStore();
|
||||
store.getTask.mockResolvedValue({ id: "FN-001", worktree: "/tmp/wt", log: [] } as any);
|
||||
const { executor, agentStore } = makeExecutor(store, makeAgent());
|
||||
const captured = spyStep(executor);
|
||||
|
||||
const node = { id: "review", kind: "prompt", column: "review", config: { prompt: "Plain." } };
|
||||
const result = await (executor as any).runGraphCustomNode(node, { id: "FN-001" }, {}, OVERRIDE);
|
||||
|
||||
expect(result.outcome).toBe("success");
|
||||
expect(agentStore.getAgent).toHaveBeenCalledWith("agent-col");
|
||||
expect(captured.step.modelProvider).toBe("anthropic");
|
||||
expect(captured.step.modelId).toBe("claude-col");
|
||||
expect(captured.step.prompt).toContain("I am the senior reviewer.");
|
||||
expect(
|
||||
loggedLines(store).some((l) => l.includes("running as column agent 'agent-col' (override)")),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("missing column agent in registry → logged, node falls back, step still executes", async () => {
|
||||
const store = createMockStore();
|
||||
store.getTask.mockResolvedValue({ id: "FN-001", worktree: "/tmp/wt", log: [] } as any);
|
||||
|
||||
@@ -1,6 +1,64 @@
|
||||
// Column-agent coding seams: execute + step-execute sessions (plan U4,
|
||||
// R2/R3/R4/R8, KTD-2/KTD-3/KTD-5/KTD-6).
|
||||
//
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// SURFACE-ENUMERATION MATRIX AUDIT (plan U7 / FN-5893)
|
||||
//
|
||||
// The invariant is proven across mode × surface × own-settings. Every cell that
|
||||
// matters has at least one assertion in one of the five column-agent test files;
|
||||
// this block is the completeness ledger (cell → file → test). `own-present` =
|
||||
// node cfg.agentId OR complete task model pair; `own-absent` = bare.
|
||||
//
|
||||
// resolver = column-agent-resolver.test.ts (core, pure precedence)
|
||||
// custom = executor-column-agent-custom-node.test.ts
|
||||
// seams = executor-column-agent-seams.test.ts (this file)
|
||||
// princ = executor-column-agent-principal.test.ts
|
||||
//
|
||||
// SURFACE: custom node ───────────────────────────────────────────────────────
|
||||
// override × own-present → custom "override column: node with own cfg.agentId…"
|
||||
// override × own-absent → custom "override column: bare node adopts the column agent"
|
||||
// defer × own-present → custom "defer column: node with own cfg.agentId keeps it…" (a)
|
||||
// defer × own-absent → custom "defer column: …bare node adopts the column agent" (b)
|
||||
//
|
||||
// SURFACE: execute seam ──────────────────────────────────────────────────────
|
||||
// override × own-present → seams "override column, task assigned to Y → …X's model"
|
||||
// override × own-absent → seams "override column, bare task (no own settings) → column agent"
|
||||
// defer × own-present → seams "defer column, task with complete modelProvider/modelId…"
|
||||
// defer × own-absent → seams "defer column, bare task (no own settings) → column agent adopted"
|
||||
//
|
||||
// SURFACE: step-execute ───────────────────────────────────────────────────────
|
||||
// override × own-present → seams "foreach instance node inherits the foreach's bound column…"
|
||||
// override × own-absent → seams "step-execute override, bare task → column-agent attribution"
|
||||
// defer × own-present → seams "defer column with task own complete model pair → assigned attribution"
|
||||
// defer × own-absent → seams "step-execute defer, bare task → column-agent attribution adopted"
|
||||
//
|
||||
// SURFACE: heartbeat-deferred (principal) ─────────────────────────────────────
|
||||
// override × own-present → princ "override column X (allowParallelExecution=false)…defers"
|
||||
// + princ "resumeTaskForAgent…pass 2 re-dispatches it"
|
||||
// override × own-absent → princ "two tasks bound to different column agents…" (bare tasks)
|
||||
// defer × own-present → princ "defer column with task own complete model pair → X NOT effective, pass 2 does not fire"
|
||||
// defer × own-absent → resolver "defer × bare → column agent wins" (gate input);
|
||||
// the deferral gate consumes resolveEffectivePrincipalId, exercised override-side above
|
||||
//
|
||||
// SURFACE: missing-agent fallback ─────────────────────────────────────────────
|
||||
// custom (override) → custom "missing column agent in registry → logged, node falls back…"
|
||||
// execute seam → seams "column agent missing from registry at seam time → fallback…"
|
||||
// step-execute → seams "column agent missing from registry at step-execute seam → fallback…"
|
||||
// restart watcher → princ "column agent deleted mid-session → no restart storm…fallback (R8)"
|
||||
//
|
||||
// NO-BINDING (parity / invisibility) ──────────────────────────────────────────
|
||||
// execute seam → seams characterization "execute seam: …assigned agent, no column-agent log"
|
||||
// step-execute → seams characterization "step session: attribution falls back to assignedAgentId"
|
||||
// gating principal → princ "no binding → gating context built for the assigned Y (byte-identical)"
|
||||
// default workflow → workflow-graph-executor-parity.test.ts "column agent feature is invisible…"
|
||||
//
|
||||
// Cells deliberately NOT separately pinned: defer × heartbeat × own-absent at the
|
||||
// *surface* level — the deferral gate's only column-agent input is the resolver
|
||||
// verdict (proven in resolver) routed through resolveEffectivePrincipalId (proven
|
||||
// override-side, where the principal differs from assignedAgentId; under defer ×
|
||||
// own-absent the principal is still the column agent by the same code path).
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
//
|
||||
// The graph EXECUTE seam (single coding session) and STEP-EXECUTE seam
|
||||
// (StepSessionExecutor per-step sessions) must run as the column agent when the
|
||||
// governing seam node's DECLARED column carries a binding. Session identity =
|
||||
@@ -273,6 +331,26 @@ describe("column-agent coding seams (plan U4)", () => {
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("override column, bare task (no own settings) → column agent adopted", async () => {
|
||||
// override × own-absent at the execute seam: the column agent wins
|
||||
// regardless of own settings, and here there are none to begin with.
|
||||
const store = createMockStore();
|
||||
const task = singleSessionTask(); // no assignedAgentId, no model pair
|
||||
store.getTask.mockResolvedValue(task as any);
|
||||
const { executor, agentStore } = makeExecutor(store, { "agent-col": makeColumnAgent() });
|
||||
installTaskDoneAgent();
|
||||
|
||||
await runExecuteSeam(executor, task, "execute-node", OVERRIDE_COL);
|
||||
|
||||
const opts = lastFnAgentOpts();
|
||||
expect(opts.defaultProvider).toBe("anthropic");
|
||||
expect(opts.defaultModelId).toBe("claude-col");
|
||||
expect(agentStore.getAgent).toHaveBeenCalledWith("agent-col");
|
||||
expect(
|
||||
loggedLines(store).some((l) => l.includes("running as column agent 'agent-col' (override)")),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("column agent missing from registry at seam time → fallback to assignedAgentId path, logged, run proceeds", async () => {
|
||||
const store = createMockStore();
|
||||
const task = singleSessionTask({ assignedAgentId: "agent-Y" });
|
||||
@@ -376,6 +454,42 @@ describe("column-agent coding seams (plan U4)", () => {
|
||||
expect(agentStore.getAgent).not.toHaveBeenCalledWith("agent-col");
|
||||
});
|
||||
|
||||
it("override, bare task (no own settings) → step session carries column-agent attribution", async () => {
|
||||
// override × own-absent at the step-execute seam.
|
||||
const store = createMockStore();
|
||||
const task = singleSessionTask(); // no assignedAgentId, no model pair
|
||||
store.getTask.mockResolvedValue(task as any);
|
||||
const { executor, agentStore } = makeExecutor(store, { "agent-col": makeColumnAgent() });
|
||||
installTaskDoneAgent();
|
||||
|
||||
await runStepSessionSeam(executor, task, "foreach-1#0:step-exec", OVERRIDE_COL);
|
||||
|
||||
const opts = lastStepExecutorOpts();
|
||||
expect(opts.effectiveAgentId).toBe("agent-col");
|
||||
expect(opts.assignedAgentRuntimeConfig).toEqual(makeColumnAgent().runtimeConfig);
|
||||
expect(agentStore.getAgent).toHaveBeenCalledWith("agent-col");
|
||||
});
|
||||
|
||||
it("defer, bare task (no own settings) → step session adopts column-agent attribution", async () => {
|
||||
// defer × own-absent at the step-execute seam: nothing suppresses defer, so
|
||||
// the column agent is adopted.
|
||||
const store = createMockStore();
|
||||
const task = singleSessionTask(); // no assignedAgentId, no model pair
|
||||
store.getTask.mockResolvedValue(task as any);
|
||||
const { executor, agentStore } = makeExecutor(store, { "agent-col": makeColumnAgent() });
|
||||
installTaskDoneAgent();
|
||||
|
||||
await runStepSessionSeam(executor, task, "foreach-1#0:step-exec", DEFER_COL);
|
||||
|
||||
const opts = lastStepExecutorOpts();
|
||||
expect(opts.effectiveAgentId).toBe("agent-col");
|
||||
expect(opts.assignedAgentRuntimeConfig).toEqual(makeColumnAgent().runtimeConfig);
|
||||
expect(agentStore.getAgent).toHaveBeenCalledWith("agent-col");
|
||||
expect(
|
||||
loggedLines(store).some((l) => l.includes("running as column agent 'agent-col' (defer)")),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("column agent missing from registry at step-execute seam → fallback to assigned-agent attribution, logged", async () => {
|
||||
const store = createMockStore();
|
||||
const task = singleSessionTask({ assignedAgentId: "agent-Y" });
|
||||
|
||||
@@ -10,7 +10,13 @@
|
||||
// suite `stepwise-workflow-parity.test.ts`. Keep the two concerns separate.
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import type { TaskDetail } from "@fusion/core";
|
||||
import type { TaskDetail, WorkflowIrV2 } from "@fusion/core";
|
||||
import {
|
||||
BUILTIN_CODING_WORKFLOW_IR,
|
||||
buildWorkflowObservation,
|
||||
buildWorkflowObservationFromTask,
|
||||
compareWorkflowRunObservations,
|
||||
} from "@fusion/core";
|
||||
|
||||
import { WorkflowGraphExecutor } from "../workflow-graph-executor.js";
|
||||
import type { WorkflowLegacySeams } from "../workflow-node-handlers.js";
|
||||
@@ -123,3 +129,73 @@ describe("WorkflowGraphExecutor interpreter-parity", () => {
|
||||
expect(seams.merge).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// COLUMN-AGENT INVISIBILITY PARITY (plan U7 / R9)
|
||||
//
|
||||
// The per-column agent feature must be invisible when no column carries a
|
||||
// binding: the built-in default workflow synthesizes no `agent` field on any
|
||||
// column, and a binding-free run produces observations identical to legacy via
|
||||
// the same `compareWorkflowRunObservations` machinery the dual-observe gate uses.
|
||||
// This is the byte-identity / parity oracle for the feature being unbound.
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
describe("column-agent feature is invisible when unbound (U7 / R9)", () => {
|
||||
it("the default built-in workflow synthesizes NO column agent field on any column", () => {
|
||||
const ir = BUILTIN_CODING_WORKFLOW_IR as WorkflowIrV2;
|
||||
expect(ir.version).toBe("v2");
|
||||
expect(ir.columns.length).toBeGreaterThan(0);
|
||||
for (const col of ir.columns) {
|
||||
// Absent, not `null` and not an explicit default — R9 omission guarantee.
|
||||
expect("agent" in col).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("a binding-free run yields observations identical to legacy (compareWorkflowRunObservations agrees)", async () => {
|
||||
// Drive the graph executor over the default execute→review→merge sequence and
|
||||
// collect the stage transitions; with zero column bindings, the column-agent
|
||||
// feature contributes nothing, so the interpreter observation must equal the
|
||||
// legacy authoritative observation with no drift.
|
||||
const stages: string[] = [];
|
||||
const seams: WorkflowLegacySeams = {
|
||||
planning: async () => ({ outcome: "success" }),
|
||||
execute: async () => ({ outcome: "success" }),
|
||||
review: async () => ({ outcome: "success" }),
|
||||
merge: async () => ({ outcome: "success" }),
|
||||
schedule: async () => ({ outcome: "success" }),
|
||||
};
|
||||
type BaseSeam = "planning" | "execute" | "review" | "merge" | "schedule";
|
||||
const executor = new WorkflowGraphExecutor({
|
||||
seams,
|
||||
handlers: {
|
||||
prompt: async (node, ctx) => {
|
||||
const seam = String(node.config?.seam) as BaseSeam;
|
||||
stages.push(seam);
|
||||
return seams[seam](ctx.task, ctx.context);
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const result = await executor.run(task, {
|
||||
experimentalFeatures: { workflowGraphExecutor: true },
|
||||
});
|
||||
expect(result.outcome).toBe("success");
|
||||
|
||||
// Legacy authoritative observation: a clean run that lands in `done`/merged.
|
||||
const legacyObs = buildWorkflowObservationFromTask(
|
||||
{ column: "done", status: "done", review: { verdict: "approve" } },
|
||||
{ columnSequence: ["todo", "in-progress", "in-review", "done"] },
|
||||
);
|
||||
// Interpreter (binding-free) observation assembled from the same run.
|
||||
const interpreterObs = buildWorkflowObservation({
|
||||
stageTransitions: ["triage", "execute", "review", "merge"],
|
||||
terminalColumn: "done",
|
||||
terminalStatus: "done",
|
||||
reviewVerdict: "approve",
|
||||
mergeOutcome: "merged",
|
||||
});
|
||||
|
||||
const report = compareWorkflowRunObservations(legacyObs, interpreterObs);
|
||||
expect(report.agree).toBe(true);
|
||||
expect(report.diffs).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user