FN-7758: route Grok CLI fallback models through the CLI runtime when no API key is visible
Fix Grok CLI 'missing API key' false-positive by extending the no-visible-key routing invariant to configured fallback models, not just the default provider. - deriveGrokRuntimeHintForNoVisibleKey now also triggers when the fallback provider is grok-cli - new applyGrokCliNoKeyRuntimeOptions promotes a grok-cli fallback model into the primary session (stripping the provider-qualified model prefix) instead of leaving it as an unused fallback - docs/grok-cli-contract.md updated to describe the fallback-routing contract - added regression tests in grok-runtime-routing.test.ts, chat-manager.test.ts, and chat-manager-room-hybrid.test.ts - added changeset fn-7758-grok-cli-no-key-routing.md (patch) Files changed: .changeset/fn-7758-grok-cli-no-key-routing.md | 7 ++ docs/grok-cli-contract.md | 29 +++-- .../src/__tests__/chat-manager-room-hybrid.test.ts | 36 +++++++ .../dashboard/src/__tests__/chat-manager.test.ts | 41 +++++++ packages/dashboard/src/chat.ts | 16 ++- .../src/__tests__/grok-runtime-routing.test.ts | 118 +++++++++++++++++++++ packages/engine/src/agent-session-helpers.ts | 36 ++++++- 7 files changed, 266 insertions(+), 17 deletions(-) Fusion-Task-Id: FN-7758 Fusion-Task-Lineage: 6b564b21-50ab-4c3e-b0d7-7fded2091d90 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
7
.changeset/fn-7758-grok-cli-no-key-routing.md
Normal file
7
.changeset/fn-7758-grok-cli-no-key-routing.md
Normal file
@@ -0,0 +1,7 @@
|
||||
---
|
||||
"@runfusion/fusion": patch
|
||||
---
|
||||
|
||||
summary: Route no-key Grok CLI chat and fallback model selections through the bundled CLI runtime.
|
||||
category: fix
|
||||
dev: Extends grok-cli no-visible-key routing to dashboard chat defaults, room responders, and fallback models.
|
||||
@@ -134,7 +134,7 @@ Notes:
|
||||
`GrokCliProviderCard.tsx`) is out of scope for this task and is not
|
||||
modified here.
|
||||
|
||||
## Wiring (resolved — FN-7725, extended by FN-7753)
|
||||
## Wiring (resolved — FN-7725, extended by FN-7753/FN-7758)
|
||||
|
||||
<!--
|
||||
FNXC:GrokCli 2026-07-09-00:00:
|
||||
@@ -160,17 +160,25 @@ via the dashboard's agent **Runtime Source → Runtime** picker
|
||||
plugin runtime, including the bundled Grok Runtime plugin's `runtimeId:
|
||||
"grok"`, with no Grok-specific code required).
|
||||
|
||||
**Automatic no-key fallback (FN-7753):** when `createResolvedAgentSession()` sees
|
||||
**Automatic no-key fallback (FN-7753/FN-7758):** when `createResolvedAgentSession()` sees
|
||||
all of the following, it derives the same effective `runtimeHint: "grok"` before
|
||||
calling `resolveRuntime()`:
|
||||
|
||||
1. no explicit runtime hint was supplied (explicit hints, including `"pi"`,
|
||||
always win);
|
||||
2. the resolved execution provider is `grok-cli`;
|
||||
2. the resolved primary/default provider is `grok-cli`, or the configured
|
||||
fallback provider is `grok-cli`;
|
||||
3. Fusion cannot see a non-empty `GROK_API_KEY` either in the environment or in
|
||||
`~/.grok/user-settings.json`'s `apiKey` field; and
|
||||
4. the bundled Grok Runtime plugin has registered runtime id `"grok"`.
|
||||
|
||||
FN-7758 also requires dashboard Chat/QuickChat and room responders to forward
|
||||
the configured default provider/model into this same session seam when a send has
|
||||
no explicit model and no bound agent runtime model. That keeps the no-key routing
|
||||
invariant identical across executor, reviewer/validator/merger-adjacent, single
|
||||
chat, QuickChat, and room responder surfaces instead of letting model-less chat
|
||||
bypass the auto-derive by omitting `defaultProvider`.
|
||||
|
||||
If a Fusion-visible key exists, the direct xAI OpenAI-compatible endpoint remains
|
||||
the default. If the Grok runtime is not registered, Fusion leaves the session on
|
||||
the existing PI/direct path rather than inventing a separate routing mode.
|
||||
@@ -207,13 +215,14 @@ additive change," formalizing + testing + documenting the already-working
|
||||
path is lower risk and closes the actual gap (an *exercised* path, not just
|
||||
an implemented adapter) without adding new user-facing config surface.
|
||||
|
||||
**Model plumbing (FN-7753):** for the automatic no-key fallback, the selected
|
||||
`grok-cli/*` model id is preserved through `AgentRuntimeOptions.defaultModelId`,
|
||||
normalized by stripping a leading `grok-cli/` (or `grok/`) prefix, and passed to
|
||||
the CLI as `grok --model <id>` alongside `--prompt` and `--format json`.
|
||||
Runtime-mode remains model-agnostic when chosen explicitly from the dashboard;
|
||||
that no-model path still uses the adapter's historical `"grok/default"` session
|
||||
fallback and omits `--model`.
|
||||
**Model plumbing (FN-7753/FN-7758):** for the automatic no-key fallback, the selected
|
||||
`grok-cli/*` model id is preserved through `AgentRuntimeOptions.defaultModelId`
|
||||
(or promoted from `fallbackModelId` when the fallback provider is the grok-cli
|
||||
selection), normalized by stripping a leading `grok-cli/` (or `grok/`) prefix,
|
||||
and passed to the CLI as `grok --model <id>` alongside `--prompt` and
|
||||
`--format json`. Runtime-mode remains model-agnostic when chosen explicitly from
|
||||
the dashboard; that no-model path still uses the adapter's historical
|
||||
`"grok/default"` session fallback and omits `--model`.
|
||||
|
||||
**Why the direct xAI endpoint stays default:** a `grok-cli/*` **model** selection
|
||||
continues to route through the direct xAI OpenAI-compatible endpoint
|
||||
|
||||
@@ -110,6 +110,42 @@ describe("ChatManager room hybrid responder resolution", () => {
|
||||
expect(result.ambient.map((agent: any) => agent.id)).toEqual(["agent-a"]);
|
||||
});
|
||||
|
||||
it("passes configured default grok-cli model to model-less room responders", async () => {
|
||||
mockChatStore.listRoomMembers.mockReturnValue([
|
||||
{ roomId: "room-1", agentId: "agent-a", role: "member", addedAt: "2026-01-01" },
|
||||
]);
|
||||
mockAgentStore.listAgents.mockResolvedValue([{ id: "agent-a", name: "Alpha", role: "executor", runtimeConfig: {} }]);
|
||||
let createOptions: any;
|
||||
|
||||
__setCreateResolvedAgentSession(async (options: any) => {
|
||||
createOptions = options;
|
||||
return {
|
||||
session: {
|
||||
prompt: vi.fn(),
|
||||
dispose: vi.fn(),
|
||||
state: {
|
||||
messages: [{ role: "assistant", content: "Room reply" }],
|
||||
},
|
||||
},
|
||||
} as any;
|
||||
});
|
||||
|
||||
const manager = new ChatManager(
|
||||
mockChatStore as any,
|
||||
"/tmp",
|
||||
mockAgentStore as any,
|
||||
undefined,
|
||||
async () => ({ defaultProvider: "grok-cli", defaultModelId: "grok-cli/grok-4.5" }),
|
||||
);
|
||||
await manager.sendRoomMessage("room-1", "hello room");
|
||||
|
||||
expect(createOptions).toMatchObject({
|
||||
sessionPurpose: "heartbeat",
|
||||
defaultProvider: "grok-cli",
|
||||
defaultModelId: "grok-cli/grok-4.5",
|
||||
});
|
||||
});
|
||||
|
||||
it("persists assistant room replies for resolved responders", async () => {
|
||||
mockChatStore.listRoomMembers.mockReturnValue([
|
||||
{ roomId: "room-1", agentId: "agent-a", role: "member", addedAt: "2026-01-01" },
|
||||
|
||||
@@ -201,6 +201,47 @@ describe("ChatManager.sendMessage", () => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it("routes model-less QuickChat through configured default grok-cli provider", async () => {
|
||||
let createOptions: any;
|
||||
__setCreateResolvedAgentSession(async (options: any) => {
|
||||
createOptions = options;
|
||||
return {
|
||||
session: {
|
||||
prompt: vi.fn().mockResolvedValue(undefined),
|
||||
dispose: vi.fn(),
|
||||
model: { provider: "grok-cli", id: "grok-4.5" },
|
||||
state: { messages: [{ role: "assistant", content: "Grok response" }] },
|
||||
},
|
||||
runtimeId: "grok",
|
||||
wasConfigured: true,
|
||||
} as any;
|
||||
});
|
||||
mockChatStore.getSession.mockReturnValue({
|
||||
id: "chat-001",
|
||||
status: "active",
|
||||
projectId: "project-a",
|
||||
});
|
||||
mockChatStore.addMessage.mockImplementation((_sessionId, input) => ({
|
||||
id: input.role === "assistant" ? "assistant-msg" : "user-msg",
|
||||
sessionId: "chat-001",
|
||||
role: input.role,
|
||||
content: input.content,
|
||||
createdAt: "2026-07-09T00:00:00.000Z",
|
||||
}));
|
||||
|
||||
const chatManager = createChatManagerWithSettings({
|
||||
defaultProvider: "grok-cli",
|
||||
defaultModelId: "grok-cli/grok-4.5",
|
||||
});
|
||||
await chatManager.sendMessage("chat-001", "Hello Grok");
|
||||
|
||||
expect(createOptions).toMatchObject({
|
||||
sessionPurpose: "executor",
|
||||
defaultProvider: "grok-cli",
|
||||
defaultModelId: "grok-cli/grok-4.5",
|
||||
});
|
||||
});
|
||||
|
||||
it("records successful chat session token usage from provider stats", async () => {
|
||||
__setCreateResolvedAgentSession(async () => ({
|
||||
session: {
|
||||
|
||||
@@ -1708,9 +1708,13 @@ export class ChatManager {
|
||||
const roomPrompt = roomPromptParts.join("\n\n");
|
||||
|
||||
const responderRuntimeModel = extractRuntimeModel(input.responder.runtimeConfig);
|
||||
const effectiveModelProvider = input.modelProvider ?? responderRuntimeModel.provider;
|
||||
const effectiveModelId = input.modelId ?? responderRuntimeModel.modelId;
|
||||
const chatModelSettings = await this.getChatModelSettings();
|
||||
/*
|
||||
* FNXC:GrokCliRouting 2026-07-09-22:10:
|
||||
* Room responders with no explicit send-time or responder runtime model still need the configured chat/project default to reach createResolvedAgentSession. Without forwarding a defaultProvider of grok-cli, the no-visible-key auto-derive seam cannot route to the Grok CLI runtime and pi can surface the direct xAI missing-key error.
|
||||
*/
|
||||
const effectiveModelProvider = input.modelProvider ?? responderRuntimeModel.provider ?? chatModelSettings.defaultProvider;
|
||||
const effectiveModelId = input.modelId ?? responderRuntimeModel.modelId ?? chatModelSettings.defaultModelId;
|
||||
/*
|
||||
* FNXC:ChatModels 2026-07-01-16:42:
|
||||
* Room responders should pass configured fallback models even when the room send chose an explicit model. The engine still swaps only for retryable provider/model-selection failures, so an unavailable Sonnet 5 can recover without making ordinary prompt errors ambiguous.
|
||||
@@ -2177,6 +2181,14 @@ export class ChatManager {
|
||||
}
|
||||
|
||||
const chatModelSettings = await this.getChatModelSettings();
|
||||
/*
|
||||
* FNXC:GrokCliRouting 2026-07-09-22:10:
|
||||
* Model-less Chat/QuickChat sessions must pass the configured default model into the shared engine session helper. This keeps grok-cli defaults on the Grok CLI runtime when Fusion has no visible key instead of bypassing FN-7753/FN-7758 routing by omitting defaultProvider entirely.
|
||||
*/
|
||||
effectiveModelProvider ??= chatModelSettings.defaultProvider;
|
||||
effectiveModelId ??= chatModelSettings.defaultModelId;
|
||||
failureContextProvider = effectiveModelProvider;
|
||||
failureContextModelId = effectiveModelId;
|
||||
const usesConfiguredDefaultModel =
|
||||
requestedModelProvider === chatModelSettings.defaultProvider
|
||||
&& requestedModelId === chatModelSettings.defaultModelId
|
||||
|
||||
@@ -318,6 +318,124 @@ describe("Grok CLI runtime routing (FN-7725)", () => {
|
||||
}));
|
||||
});
|
||||
|
||||
it("auto-routes heartbeat/room responder grok-cli defaults to the Grok runtime when no Fusion-visible key exists", async () => {
|
||||
vi.mocked(fusionCore.isGrokApiKeyFusionVisible).mockReturnValue(false);
|
||||
const spawn = vi.fn().mockReturnValue(makeFakeGrokProcess().proc);
|
||||
const grokRegistration = await createGrokRegistration(spawn);
|
||||
const pluginRunner = createMockPluginRunner({
|
||||
getRuntimeById: vi.fn().mockReturnValue(grokRegistration),
|
||||
});
|
||||
const audit = { database: vi.fn().mockResolvedValue(undefined) };
|
||||
|
||||
const result = await createResolvedAgentSession({
|
||||
sessionPurpose: "heartbeat",
|
||||
pluginRunner,
|
||||
runAuditor: audit as never,
|
||||
cwd: "/tmp/project",
|
||||
defaultProvider: "grok-cli",
|
||||
defaultModelId: "grok-4.5",
|
||||
systemPrompt: "room-responder",
|
||||
});
|
||||
|
||||
expect(result.runtimeId).toBe("grok");
|
||||
expect(result.wasConfigured).toBe(true);
|
||||
expect(result.session).toMatchObject({ model: "grok-4.5" });
|
||||
expect(audit.database).toHaveBeenCalledWith(expect.objectContaining({
|
||||
type: "session:runtime-resolved",
|
||||
target: "grok",
|
||||
metadata: expect.objectContaining({
|
||||
sessionPurpose: "heartbeat",
|
||||
runtimeHint: "grok",
|
||||
reason: "grok-cli-no-visible-key",
|
||||
}),
|
||||
}));
|
||||
});
|
||||
|
||||
it("auto-routes a grok-cli fallback model to the Grok runtime when no Fusion-visible key exists", async () => {
|
||||
vi.mocked(fusionCore.isGrokApiKeyFusionVisible).mockReturnValue(false);
|
||||
const spawn = vi.fn().mockReturnValue(makeFakeGrokProcess().proc);
|
||||
const grokRegistration = await createGrokRegistration(spawn);
|
||||
const pluginRunner = createMockPluginRunner({
|
||||
getRuntimeById: vi.fn().mockReturnValue(grokRegistration),
|
||||
});
|
||||
const audit = { database: vi.fn().mockResolvedValue(undefined) };
|
||||
|
||||
const result = await createResolvedAgentSession({
|
||||
sessionPurpose: "executor",
|
||||
pluginRunner,
|
||||
runAuditor: audit as never,
|
||||
cwd: "/tmp/project",
|
||||
defaultProvider: "openai",
|
||||
defaultModelId: "gpt-4o",
|
||||
fallbackProvider: "grok-cli",
|
||||
fallbackModelId: "grok-cli/grok-4.5",
|
||||
systemPrompt: "fallback-selection-only",
|
||||
});
|
||||
|
||||
expect(result.runtimeId).toBe("grok");
|
||||
expect(result.wasConfigured).toBe(true);
|
||||
expect(mockCreateFnAgent).not.toHaveBeenCalled();
|
||||
expect(result.session).toMatchObject({ model: "grok-4.5" });
|
||||
expect(audit.database).toHaveBeenCalledWith(expect.objectContaining({
|
||||
type: "session:runtime-resolved",
|
||||
target: "grok",
|
||||
metadata: expect.objectContaining({
|
||||
runtimeHint: "grok",
|
||||
reason: "grok-cli-no-visible-key",
|
||||
provider: "openai",
|
||||
modelId: "gpt-4o",
|
||||
}),
|
||||
}));
|
||||
});
|
||||
|
||||
it("auto-routes a bare grok-cli fallback model id without adding a provider prefix", async () => {
|
||||
vi.mocked(fusionCore.isGrokApiKeyFusionVisible).mockReturnValue(false);
|
||||
const spawn = vi.fn().mockReturnValue(makeFakeGrokProcess().proc);
|
||||
const grokRegistration = await createGrokRegistration(spawn);
|
||||
const pluginRunner = createMockPluginRunner({
|
||||
getRuntimeById: vi.fn().mockReturnValue(grokRegistration),
|
||||
});
|
||||
|
||||
const result = await createResolvedAgentSession({
|
||||
sessionPurpose: "validation",
|
||||
pluginRunner,
|
||||
cwd: "/tmp/project",
|
||||
defaultProvider: "anthropic",
|
||||
defaultModelId: "claude-sonnet-4-5",
|
||||
fallbackProvider: "grok-cli",
|
||||
fallbackModelId: "grok-4.5",
|
||||
systemPrompt: "fallback-bare-model",
|
||||
});
|
||||
|
||||
expect(result.runtimeId).toBe("grok");
|
||||
expect(result.session).toMatchObject({ model: "grok-4.5" });
|
||||
});
|
||||
|
||||
it("keeps mock/test-mode provider routing on the mock runtime when grok-cli fallback is configured", async () => {
|
||||
vi.mocked(fusionCore.isGrokApiKeyFusionVisible).mockReturnValue(false);
|
||||
const spawn = vi.fn().mockReturnValue(makeFakeGrokProcess().proc);
|
||||
const grokRegistration = await createGrokRegistration(spawn);
|
||||
const pluginRunner = createMockPluginRunner({
|
||||
getRuntimeById: vi.fn().mockReturnValue(grokRegistration),
|
||||
});
|
||||
|
||||
const result = await createResolvedAgentSession({
|
||||
sessionPurpose: "executor",
|
||||
pluginRunner,
|
||||
cwd: "/tmp/project",
|
||||
defaultProvider: "mock",
|
||||
defaultModelId: "scripted",
|
||||
fallbackProvider: "grok-cli",
|
||||
fallbackModelId: "grok-4.5",
|
||||
systemPrompt: "mock-mode",
|
||||
});
|
||||
|
||||
expect(result.runtimeId).toBe("mock");
|
||||
expect(result.wasConfigured).toBe(true);
|
||||
expect(pluginRunner.getRuntimeById).not.toHaveBeenCalledWith("grok");
|
||||
expect(mockCreateFnAgent).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("honors explicit runtime hints over the no-key grok-cli auto-derivation", async () => {
|
||||
vi.mocked(fusionCore.isGrokApiKeyFusionVisible).mockReturnValue(false);
|
||||
const spawn = vi.fn().mockReturnValue(makeFakeGrokProcess().proc);
|
||||
|
||||
@@ -167,7 +167,8 @@ function deriveGrokRuntimeHintForNoVisibleKey(
|
||||
runtimeOptions: AgentRuntimeOptions,
|
||||
pluginRunner: PluginRunner | undefined,
|
||||
): string | undefined {
|
||||
if (runtimeOptions.defaultProvider !== GROK_CLI_PROVIDER_ID) return undefined;
|
||||
if (runtimeOptions.defaultProvider !== GROK_CLI_PROVIDER_ID
|
||||
&& runtimeOptions.fallbackProvider !== GROK_CLI_PROVIDER_ID) return undefined;
|
||||
if (isGrokApiKeyFusionVisible()) return undefined;
|
||||
try {
|
||||
return pluginRunner?.getRuntimeById("grok") ? "grok" : undefined;
|
||||
@@ -176,6 +177,29 @@ function deriveGrokRuntimeHintForNoVisibleKey(
|
||||
}
|
||||
}
|
||||
|
||||
function applyGrokCliNoKeyRuntimeOptions(
|
||||
runtimeOptions: AgentRuntimeOptions,
|
||||
): AgentRuntimeOptions {
|
||||
if (runtimeOptions.defaultProvider === GROK_CLI_PROVIDER_ID) {
|
||||
return {
|
||||
...runtimeOptions,
|
||||
defaultModelId: stripGrokCliModelProviderPrefix(runtimeOptions.defaultModelId),
|
||||
};
|
||||
}
|
||||
|
||||
if (runtimeOptions.fallbackProvider === GROK_CLI_PROVIDER_ID) {
|
||||
return {
|
||||
...runtimeOptions,
|
||||
defaultProvider: runtimeOptions.fallbackProvider,
|
||||
defaultModelId: stripGrokCliModelProviderPrefix(runtimeOptions.fallbackModelId),
|
||||
fallbackProvider: undefined,
|
||||
fallbackModelId: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
return runtimeOptions;
|
||||
}
|
||||
|
||||
function pickSettingsThenRuntimeModel(
|
||||
settingsModel: ResolvedModelSelection,
|
||||
assignedAgentRuntimeConfig?: Record<string, unknown>,
|
||||
@@ -406,16 +430,18 @@ export async function createResolvedAgentSession(
|
||||
Explicit runtime hints always win, visible keys keep the direct xAI endpoint default, and mock/test-mode
|
||||
provider routing stays on the mock runtime. Strip only the provider-qualified model prefix so the CLI
|
||||
receives the concrete selected model via GrokRuntimeAdapter without changing non-grok sessions.
|
||||
|
||||
FNXC:GrokCliRouting 2026-07-09-22:10:
|
||||
FN-7758 extends the no-visible-key invariant to configured fallback models. Pi resolves fallback models
|
||||
during session creation and prompt-time swaps through the key-requiring provider registry, so a grok-cli
|
||||
fallback must select the Grok CLI runtime up front and promote the fallback model into the CLI session.
|
||||
*/
|
||||
const autoGrokRuntimeHint = !useMockRuntime && !runtimeHint
|
||||
? deriveGrokRuntimeHintForNoVisibleKey(runtimeOptions, pluginRunner)
|
||||
: undefined;
|
||||
const effectiveRuntimeHint = autoGrokRuntimeHint ?? runtimeHint;
|
||||
const effectiveRuntimeOptionsWithModel: AgentRuntimeOptions = autoGrokRuntimeHint
|
||||
? {
|
||||
...effectiveRuntimeOptions,
|
||||
defaultModelId: stripGrokCliModelProviderPrefix(effectiveRuntimeOptions.defaultModelId),
|
||||
}
|
||||
? applyGrokCliNoKeyRuntimeOptions(effectiveRuntimeOptions)
|
||||
: effectiveRuntimeOptions;
|
||||
|
||||
const resolved = useMockRuntime
|
||||
|
||||
Reference in New Issue
Block a user