diff --git a/.changeset/fn-7725-grok-cli-routing.md b/.changeset/fn-7725-grok-cli-routing.md new file mode 100644 index 0000000000..cfed22b734 --- /dev/null +++ b/.changeset/fn-7725-grok-cli-routing.md @@ -0,0 +1,7 @@ +--- +"@runfusion/fusion": minor +--- + +summary: Grok work can now be routed through the Grok CLI streaming runtime, not only the direct xAI endpoint. +category: feature +dev: FN-7725 formalizes, tests, and documents the existing agent Runtime-mode picker path (option (a)) as the decided Grok CLI routing wiring — setting an agent's Runtime Source to "Runtime" -> "Grok Runtime" sets runtimeConfig.runtimeHint="grok", which the existing generic extractRuntimeHint -> resolveRuntime -> resolvePluginRuntime -> plugin factory chain (packages/engine/src/agent-session-helpers.ts, runtime-resolution.ts) already resolved to GrokRuntimeAdapter (FN-7722) for other plugin runtimes; no new engine/dashboard code was required, only a routing test (packages/engine/src/__tests__/grok-runtime-routing.test.ts), an FNXC decision note at the extractRuntimeHint seam, and documentation. Direct xAI OpenAI-compatible path (FN-7711/FN-7714) remains the default and is unchanged; the new path is additive/opt-in and does not preserve a specific grok-cli/* model selection (documented limitation). Contract decision recorded in docs/grok-cli-contract.md. diff --git a/docs/grok-cli-contract.md b/docs/grok-cli-contract.md index 2739ca32d0..316b46372f 100644 --- a/docs/grok-cli-contract.md +++ b/docs/grok-cli-contract.md @@ -134,19 +134,78 @@ Notes: `GrokCliProviderCard.tsx`) is out of scope for this task and is not modified here. -## Wiring gap (recorded, not closed by this task) +## Wiring (resolved — FN-7725) -`packages/engine/src/runtime-resolution.ts`'s `resolveRuntime()` only reaches -a plugin runtime adapter (like `GrokRuntimeAdapter`) when -`runtimeConfig.runtimeHint === "grok"`. A repo-wide grep at Step 0 of this -task confirmed **nothing in the product sets `runtimeHint` to `"grok"` -today** (task/agent config, settings, or otherwise) — the same wiring gap -FN-7715's stale comment already noted. This task lands the adapter -implementation and its tests, but does **not** wire an end-to-end path that -exercises it (no product code sets `runtimeHint: "grok"`, and no settings -toggle exists to prefer CLI execution over the direct endpoint). That wiring -is filed as a follow-up task (see `fn_task_create` entries linked from this -task). + + +**Decision: option (a) — formalize, document, and test the existing agent +Runtime-mode picker path. Do NOT add a new settings toggle (option (b)).** + +**Trigger:** an agent's `runtimeConfig.runtimeHint === "grok"`, set today via +the dashboard's agent **Runtime Source → Runtime** picker +(`NewAgentDialog.tsx` / `AgentDetailView.tsx`), which is populated from +`GET /api/plugins/runtimes` (already generic — surfaces every registered +plugin runtime, including the bundled Grok Runtime plugin's `runtimeId: +"grok"`, with no Grok-specific code required). + +**Exact seam:** `packages/engine/src/agent-session-helpers.ts`'s +`extractRuntimeHint(runtimeConfig)` reads that hint from the assigned agent's +`runtimeConfig` and threads it, as `runtimeHint`, into +`packages/engine/src/runtime-resolution.ts`'s `resolveRuntime()` — which is +totally runtime-agnostic: when the hint matches a registered plugin +`runtimeId`, `resolvePluginRuntime()` calls that plugin's `runtime.factory` +(the Grok plugin's factory returns `new GrokRuntimeAdapter()`, +`plugins/fusion-plugin-grok-runtime/src/index.ts`) and the resolved adapter +becomes the session's runtime. This same generic chain already carries +`"hermes"` and `"droid"` runtime hints end-to-end (see +`hermes-runtime-integration.test.ts`, `droid-runtime-e2e.test.ts`) — Grok's +plugin registration alone was sufficient for the chain to reach it; **no +engine or dashboard code changed for this task**, because the generic +Runtime-mode picker → `extractRuntimeHint` → `resolveRuntime` → +`resolvePluginRuntime` → plugin `factory` chain was already correct and +exercised for other plugin runtimes. FN-7725 formalizes this as the *decided* +Grok wiring, adds `packages/engine/src/__tests__/grok-runtime-routing.test.ts` +proving the chain specifically resolves `GrokRuntimeAdapter` (id `"grok"`) +and drives its `onText` streaming seam via a faked spawn (see +`runtime-adapter.ts`'s injectable `spawn` option; no live `grok` binary), and +records the decision here plus in `plugins/fusion-plugin-grok-runtime/README.md`. + +**Why option (a), not (b):** option (b) (an opt-in "prefer CLI runtime" +setting deriving `runtimeHint: "grok"` from a `grok-cli/*` model selection) +would add a new `Settings` field, defaulting/resolution logic, and a +SettingsModal UI toggle (desktop + mobile) — net-new surface area for a path +that, on inspection, was **already fully wired generically** by the existing +Runtime-mode picker. Per the Decision guidance's preference for "the smaller, +additive change," formalizing + testing + documenting the already-working +path is lower risk and closes the actual gap (an *exercised* path, not just +an implemented adapter) without adding new user-facing config surface. + +**Known limitation (by design, unchanged by this task):** Runtime-mode is +model-agnostic — `NewAgentDialog.tsx`/`AgentDetailView.tsx` clear the `model` +field when Runtime mode is selected (`model: runtimeMode === "runtime" ? "" +: ...`), so `GrokRuntimeAdapter.createSession()` never receives a +`defaultModelId` from this path and always falls back to `"grok/default"`. A +specific `grok-cli/*` model choice is therefore not preserved when routing +via Runtime-mode. Preserving model selection through the CLI runtime would +require option (b) (or an equivalent); it is filed as a follow-up task only +if genuinely warranted (see Follow-ups below), not implemented here. + +**Why the direct xAI endpoint stays default:** nothing in this task changes +what a `grok-cli/*` **model** selection does — it continues to route through +the direct xAI OpenAI-compatible endpoint (FN-7711/FN-7714, +`packages/core/src/grok-provider.ts`, `packages/engine/src/pi.ts`), which +this task does not touch. The CLI-routed path is reached *only* by the +separate, explicit agent Runtime-mode choice — an opt-in, additive, +fully-reversible path (nothing sets the hint unless an operator explicitly +picks Runtime mode for that agent). ## Decision @@ -183,18 +242,23 @@ Rationale: - FN-7716's probe/auth-readiness surface (`probe.ts`, `register-auth-routes.ts`, `GrokCliProviderCard.tsx`) is untouched by this task. -- End-to-end routing (making the product actually set - `runtimeHint === "grok"`, or adding a settings toggle to prefer the CLI - over the direct endpoint) is explicitly out of scope here and is filed as - a follow-up task. +- End-to-end routing was out of scope for FN-7722 and is resolved by + FN-7725 (see "Wiring" above): decision option (a), formalizing the + existing agent Runtime-mode picker path. No settings toggle was added. ## Follow-ups filed from this task See the task's `fn_task_create` calls (linked from FN-7722) for: -1. End-to-end routing wiring — actually setting `runtimeHint === "grok"` (or - a settings toggle preferring the CLI) so `GrokRuntimeAdapter` is - exercised in a real execution path. +1. ~~End-to-end routing wiring~~ — **closed by FN-7725** (see "Wiring" + above): the agent Runtime-mode picker path was formalized, documented, + and covered by `packages/engine/src/__tests__/grok-runtime-routing.test.ts`. 2. Full tool-call/break-early bridging for `tool_use` NDJSON events, if a future need for Grok-CLI-driven tool execution arises (out of scope for - the scoped text/no-thinking adapter landed here). + the scoped text/no-thinking adapter landed here; tracked separately as + FN-7724). +3. (Filed by FN-7725, if warranted) Preserving a specific `grok-cli/*` model + selection when routing through the CLI runtime (Runtime-mode is currently + model-agnostic — see "Known limitation" in "Wiring" above). This is the + deferred option (b) shape; only file it if a genuine operator need + surfaces, per the task's Decision guidance. diff --git a/packages/engine/src/__tests__/grok-runtime-routing.test.ts b/packages/engine/src/__tests__/grok-runtime-routing.test.ts new file mode 100644 index 0000000000..a2fb294c10 --- /dev/null +++ b/packages/engine/src/__tests__/grok-runtime-routing.test.ts @@ -0,0 +1,275 @@ +import { EventEmitter } from "node:events"; +import { PassThrough } from "node:stream"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { PluginRunner } from "../plugin-runner.js"; +import type { PluginRuntimeRegistration } from "@fusion/core"; +import { resolveRuntime } from "../runtime-resolution.js"; +import { createResolvedAgentSession, extractRuntimeHint } from "../agent-session-helpers.js"; + +/* +FNXC:GrokCli 2026-07-09-00:00: +FN-7725: end-to-end routing test for the decided wiring (option (a), +docs/grok-cli-contract.md "Wiring") — an agent's +`runtimeConfig.runtimeHint === "grok"` (as set by the dashboard's Runtime +Source -> Runtime picker) must resolve the REAL GrokRuntimeAdapter (FN-7722, +imported unmodified from the plugin package, not re-implemented/mocked here) +through the generic extractRuntimeHint -> resolveRuntime -> +resolvePluginRuntime -> plugin factory chain, and driving a prompt through +that resolved session must invoke onText from faked NDJSON `text` lines. +Uses the adapter's own injectable `spawn` seam (runtime-adapter.ts) — no +live `grok` binary, no real subprocess, no real network. Also asserts the +Surface Enumeration invariant: trigger OFF / unset / non-grok hints still +fall back to the default pi runtime unchanged, and an empty/undefined +runtimeConfig does not crash. +*/ + +const mockCreateFnAgent = vi.hoisted(() => vi.fn()); + +vi.mock("../logger.js", () => ({ + createLogger: vi.fn(() => ({ + log: vi.fn(), + warn: vi.fn(), + error: vi.fn(), + })), +})); + +vi.mock("../pi.js", () => ({ + createFnAgent: mockCreateFnAgent, + promptWithFallback: vi.fn().mockResolvedValue(undefined), + describeModel: vi.fn().mockReturnValue("pi/default"), +})); + +function grokRuntimeAdapterModulePath(): string { + return fileURLToPath( + new URL("../../../../plugins/fusion-plugin-grok-runtime/src/runtime-adapter.ts", import.meta.url), + ); +} + +type GrokRuntimeAdapterCtor = new (options?: { + binary?: string; + spawn?: (binary: string, prompt: string, options?: { cwd?: string; signal?: AbortSignal }) => unknown; +}) => { + id: string; + name: string; + createSession: (options?: unknown) => Promise<{ session: unknown; sessionFile?: string }>; + promptWithFallback: (session: unknown, prompt: string, options?: unknown) => Promise; + describeModel: (session: unknown) => string; +}; + +async function loadGrokRuntimeAdapter(): Promise { + const mod = (await import(pathToFileURL(grokRuntimeAdapterModulePath()).href)) as { + GrokRuntimeAdapter: GrokRuntimeAdapterCtor; + }; + return mod.GrokRuntimeAdapter; +} + +/** Fake `GrokStreamProcess`: an EventEmitter + a writable PassThrough stdout, matching + * the shape runtime-adapter.ts's own fixture tests use (no live subprocess). */ +function makeFakeGrokProcess(): { proc: unknown; stdout: PassThrough; kill: ReturnType } { + const stdout = new PassThrough(); + const emitter = new EventEmitter(); + const kill = vi.fn(); + const proc = Object.assign(emitter, { stdout, kill }); + return { proc, stdout, kill }; +} + +function createMockPluginRunner(overrides: Partial = {}): PluginRunner { + return { + getPluginRuntimes: vi.fn().mockReturnValue([]), + getRuntimeById: vi.fn().mockReturnValue(undefined), + createRuntimeContext: vi.fn().mockResolvedValue({ + pluginId: "fusion-plugin-grok-runtime", + taskStore: {}, + settings: {}, + logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() }, + emitEvent: vi.fn(), + }), + ...overrides, + } as unknown as PluginRunner; +} + +async function createGrokRegistration( + spawnFn: ReturnType, +): Promise<{ pluginId: string; runtime: PluginRuntimeRegistration }> { + const GrokRuntimeAdapter = await loadGrokRuntimeAdapter(); + return { + pluginId: "fusion-plugin-grok-runtime", + runtime: { + metadata: { + runtimeId: "grok", + name: "Grok Runtime", + description: "Grok CLI runtime support for Fusion", + version: "0.1.0", + }, + factory: vi.fn().mockImplementation(async () => new GrokRuntimeAdapter({ spawn: spawnFn })), + }, + }; +} + +describe("Grok CLI runtime routing (FN-7725)", () => { + beforeEach(() => { + vi.clearAllMocks(); + mockCreateFnAgent.mockResolvedValue({ + session: { runtime: "pi", prompt: vi.fn() }, + sessionFile: "/tmp/pi.session.json", + }); + }); + + afterEach(() => { + vi.restoreAllMocks(); + }); + + it("resolves the real GrokRuntimeAdapter via resolveRuntime when runtimeHint is 'grok'", async () => { + const spawn = vi.fn().mockReturnValue(makeFakeGrokProcess().proc); + const grokRegistration = await createGrokRegistration(spawn); + const pluginRunner = createMockPluginRunner({ + getRuntimeById: vi.fn().mockReturnValue(grokRegistration), + }); + + const resolved = await resolveRuntime({ + sessionPurpose: "executor", + runtimeHint: "grok", + pluginRunner, + }); + + expect(resolved.runtimeId).toBe("grok"); + expect(resolved.wasConfigured).toBe(true); + expect(resolved.runtime.id).toBe("grok"); + expect(resolved.runtime.name).toBe("Grok Runtime"); + expect(pluginRunner.getRuntimeById).toHaveBeenCalledWith("grok"); + }); + + it("createResolvedAgentSession routes an agent's runtimeConfig.runtimeHint through to GrokRuntimeAdapter and streams onText from faked NDJSON", async () => { + const { proc, stdout } = makeFakeGrokProcess(); + const spawn = vi.fn().mockReturnValue(proc); + const grokRegistration = await createGrokRegistration(spawn); + const pluginRunner = createMockPluginRunner({ + getRuntimeById: vi.fn().mockReturnValue(grokRegistration), + }); + + // Mirrors the exact seam: dashboard Runtime-mode picker writes + // agent.runtimeConfig.runtimeHint = "grok"; extractRuntimeHint reads it. + const agentRuntimeConfig = { runtimeHint: "grok" }; + const runtimeHint = extractRuntimeHint(agentRuntimeConfig); + expect(runtimeHint).toBe("grok"); + + const onText = vi.fn(); + const result = await createResolvedAgentSession({ + sessionPurpose: "executor", + runtimeHint, + pluginRunner, + cwd: "/tmp/project", + systemPrompt: "You are helpful", + onText, + }); + + expect(result.runtimeId).toBe("grok"); + expect(result.wasConfigured).toBe(true); + expect(mockCreateFnAgent).not.toHaveBeenCalled(); + + // Drive the resolved session's promptWithFallback (attached by + // createResolvedAgentSession) and feed faked NDJSON `text` lines through + // the adapter's injected fake stdout — no live grok binary involved. + const session = result.session as { promptWithFallback: (prompt: string) => Promise }; + const promptPromise = session.promptWithFallback("hello grok"); + + stdout.write(`${JSON.stringify({ type: "step_start", stepNumber: 1, timestamp: 1 })}\n`); + stdout.write(`${JSON.stringify({ type: "text", stepNumber: 1, text: "hi ", timestamp: 2 })}\n`); + stdout.write(`${JSON.stringify({ type: "text", stepNumber: 1, text: "there", timestamp: 3 })}\n`); + stdout.write( + `${JSON.stringify({ type: "step_finish", stepNumber: 1, timestamp: 4, finishReason: "stop", usage: {} })}\n`, + ); + (proc as EventEmitter).emit("close", 0, null); + + await promptPromise; + + expect(spawn).toHaveBeenCalledWith("grok", "hello grok", expect.objectContaining({})); + expect(onText.mock.calls.map((c) => c[0])).toEqual(["hi ", "there"]); + }); + + it("falls back to the default pi runtime when the Grok plugin runtime is not registered", async () => { + const pluginRunner = createMockPluginRunner({ + getRuntimeById: vi.fn().mockReturnValue(undefined), + }); + + const result = await createResolvedAgentSession({ + sessionPurpose: "executor", + runtimeHint: "grok", + pluginRunner, + cwd: "/tmp/project", + systemPrompt: "fallback", + }); + + expect(result.runtimeId).toBe("pi"); + expect(result.wasConfigured).toBe(false); + expect(mockCreateFnAgent).toHaveBeenCalledWith({ + cwd: "/tmp/project", + systemPrompt: "fallback", + }); + }); + + it("does not route through Grok when runtimeHint is unset (non-grok agent unaffected)", async () => { + const pluginRunner = createMockPluginRunner(); + + // No runtimeHint set on the agent's runtimeConfig at all. + const runtimeHint = extractRuntimeHint(undefined); + expect(runtimeHint).toBeUndefined(); + + const result = await createResolvedAgentSession({ + sessionPurpose: "executor", + runtimeHint, + pluginRunner, + cwd: "/tmp/project", + systemPrompt: "unhinted", + }); + + expect(result.runtimeId).toBe("pi"); + expect(result.wasConfigured).toBe(false); + expect(pluginRunner.getRuntimeById).not.toHaveBeenCalled(); + }); + + it("does not route through Grok for a different runtime hint (e.g. a grok-cli/* model selection stays on the default pi runtime)", async () => { + const pluginRunner = createMockPluginRunner({ + getRuntimeById: vi.fn().mockReturnValue(undefined), + }); + + // A grok-cli/* MODEL selection (Built-in Model mode) never sets + // runtimeHint at all -- it is passed as runtimeConfig.model, which + // extractRuntimeHint does not read. Simulate that shape explicitly. + const runtimeHint = extractRuntimeHint({ model: "grok-cli/grok-4" }); + expect(runtimeHint).toBeUndefined(); + + const result = await createResolvedAgentSession({ + sessionPurpose: "executor", + runtimeHint, + pluginRunner, + cwd: "/tmp/project", + systemPrompt: "model-selection-only", + }); + + expect(result.runtimeId).toBe("pi"); + expect(result.wasConfigured).toBe(false); + expect(pluginRunner.getRuntimeById).not.toHaveBeenCalled(); + }); + + it("does not crash and falls back to pi for an empty/undefined runtimeConfig", async () => { + const pluginRunner = createMockPluginRunner(); + + expect(extractRuntimeHint(undefined)).toBeUndefined(); + expect(extractRuntimeHint({})).toBeUndefined(); + expect(extractRuntimeHint({ runtimeHint: "" })).toBeUndefined(); + expect(extractRuntimeHint({ runtimeHint: 42 as unknown as string })).toBeUndefined(); + + const result = await createResolvedAgentSession({ + sessionPurpose: "executor", + runtimeHint: extractRuntimeHint({}), + pluginRunner, + cwd: "/tmp/project", + systemPrompt: "empty-config", + }); + + expect(result.runtimeId).toBe("pi"); + expect(result.wasConfigured).toBe(false); + }); +}); diff --git a/packages/engine/src/agent-session-helpers.ts b/packages/engine/src/agent-session-helpers.ts index 1186df9389..d201aafc3b 100644 --- a/packages/engine/src/agent-session-helpers.ts +++ b/packages/engine/src/agent-session-helpers.ts @@ -88,6 +88,18 @@ export interface ResolvedSessionResult { /** * Extract runtime hint from untyped runtimeConfig payload. * + * FNXC:GrokCli 2026-07-09-00:00: + * FN-7725: this is the exact seam the decided Grok CLI routing wiring (option + * (a) in docs/grok-cli-contract.md "Wiring") depends on. When an agent's + * Runtime Source is set to "Runtime" (NewAgentDialog.tsx/AgentDetailView.tsx) + * with the bundled Grok Runtime plugin selected, `runtimeConfig.runtimeHint` + * is `"grok"`, and this value flows unchanged into `resolveRuntime()` + * (runtime-resolution.ts), which resolves the Grok plugin's `GrokRuntimeAdapter` + * generically — the same chain already used by the hermes/droid plugin + * runtimes, so no Grok-specific logic lives here. The direct xAI + * OpenAI-compatible model path (grok-provider.ts) is untouched and remains + * the default; this hint-based path is opt-in and additive. + * * @param runtimeConfig - Agent/task runtime configuration * @returns normalized runtime hint or undefined when missing/invalid */ diff --git a/plugins/fusion-plugin-grok-runtime/README.md b/plugins/fusion-plugin-grok-runtime/README.md index 743e377acc..8b09dd3409 100644 --- a/plugins/fusion-plugin-grok-runtime/README.md +++ b/plugins/fusion-plugin-grok-runtime/README.md @@ -66,11 +66,43 @@ grok --prompt "" --format json OpenAI-compatible streaming path (`https://api.x.ai/v1`), which still requires one. - This adapter is only reached when an agent's - `runtimeConfig.runtimeHint === "grok"`. Nothing in the product sets that - today — routing Grok execution through the CLI end-to-end (vs. the direct - xAI endpoint, which remains the default) is tracked as a follow-up. See + `runtimeConfig.runtimeHint === "grok"`. See "Routing Grok through the CLI + runtime (FN-7725)" below for how to set that, and `docs/grok-cli-contract.md` for the full contract and decision record. +## Routing Grok through the CLI runtime (FN-7725) + +By default, selecting a `grok-cli/*` **model** for an agent/task still routes +execution through the **direct xAI OpenAI-compatible endpoint** +(`https://api.x.ai/v1`, FN-7711/FN-7714) — this default is unchanged by this +plugin. + +To route a specific agent's execution through the `grok` CLI's own +non-interactive streaming mode (`grok --prompt --format json`) instead: + +1. Open the agent in the dashboard (**New Agent** or an existing agent's + detail view). +2. Under **Runtime Source**, choose **Runtime** instead of **Built-in + Model**. +3. Select **Grok Runtime** from the runtime dropdown (sourced from + `GET /api/plugins/runtimes`, which lists every installed plugin runtime + including this one). +4. Save. The agent's `runtimeConfig.runtimeHint` is now `"grok"`; every + session that agent drives (as an assigned executor, column agent, or + child agent) resolves through `packages/engine/src/runtime-resolution.ts` + to this plugin's `GrokRuntimeAdapter` instead of the default pi runtime. + +**Known limitation:** Runtime-mode is model-agnostic — it does not carry a +specific `grok-cli/*` model id through to the adapter, so +`GrokRuntimeAdapter.createSession()` always falls back to `"grok/default"`. +If you need a specific Grok model honored end-to-end, use the direct xAI +endpoint path (**Built-in Model** → a `grok-cli/*` model) instead — that +path does preserve model selection, just not via the CLI binary. + +This routing is opt-in and per-agent; it does not change any other agent's +or task's execution path, and it does not change what a `grok-cli/*` model +selection does under **Built-in Model** mode. + ## Enable via Settings → Authentication 1. Install the `grok` CLI and authenticate it by any method it supports