FN-8098: add executor model fallback
Add workflow-specific executor fallback configuration and bounded recovery. - Add executor fallback provider, model, and thinking settings across core schemas and settings UI. - Route executor, heartbeat, child, and workflow-step sessions through the executor fallback resolver. - Retry the primary model after fallback failure before reporting terminal exhaustion. Files changed: .changeset/fn-8098-model-fallback.md | 7 +++ docs/settings-reference.md | 7 ++- .../core/src/__tests__/model-resolution.test.ts | 13 ++++ .../core/src/__tests__/settings-parity.test.ts | 5 ++ packages/core/src/builtin-workflow-settings.ts | 24 +++++++ packages/core/src/index.gate.ts | 1 + packages/core/src/index.ts | 1 + packages/core/src/model-resolution.ts | 21 +++++++ packages/core/src/settings-schema.ts | 3 + packages/core/src/types.ts | 11 ++++ .../app/components/WorkflowSettingsPanel.tsx | 8 +++ .../settings/sections/ProjectModelsSection.tsx | 10 ++- packages/engine/src/__tests__/pi.test.ts | 14 ++++- packages/engine/src/agent-session-helpers.ts | 7 ++- packages/engine/src/executor.ts | 48 +++++++------- packages/engine/src/pi.ts | 73 ++++++++-------------- packages/engine/src/step-session-executor.ts | 8 ++- 17 files changed, 180 insertions(+), 81 deletions(-) Fusion-Task-Id: FN-8098 Fusion-Task-Lineage: 61b3103b-357b-431a-8d58-411e7806b87b Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
7
.changeset/fn-8098-model-fallback.md
Normal file
7
.changeset/fn-8098-model-fallback.md
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
---
|
||||||
|
"@runfusion/fusion": minor
|
||||||
|
---
|
||||||
|
|
||||||
|
summary: Add an executor fallback model and retry the primary model before blocking on fallback exhaustion.
|
||||||
|
category: feature
|
||||||
|
dev: Adds executionFallbackProvider, executionFallbackModelId, and executionFallbackThinkingLevel workflow settings.
|
||||||
@@ -561,6 +561,9 @@ Default notes:
|
|||||||
| `defaultThinkingLevelOverride` | `ThinkingLevel` | `undefined` | Optional project default-lane thinking override used when a task does not set `thinkingLevel`; inherits `defaultThinkingLevel` when unset. |
|
| `defaultThinkingLevelOverride` | `ThinkingLevel` | `undefined` | Optional project default-lane thinking override used when a task does not set `thinkingLevel`; inherits `defaultThinkingLevel` when unset. |
|
||||||
| `executionProvider` | `string` | `undefined` | Provider for task execution agents. |
|
| `executionProvider` | `string` | `undefined` | Provider for task execution agents. |
|
||||||
| `executionModelId` | `string` | `undefined` | Model ID for task execution agents. |
|
| `executionModelId` | `string` | `undefined` | Model ID for task execution agents. |
|
||||||
|
| `executionFallbackProvider` | `string` | `undefined` | Workflow fallback provider for executor sessions; paired with `executionFallbackModelId` and resolves before the shared fallback pair. |
|
||||||
|
| `executionFallbackModelId` | `string` | `undefined` | Workflow fallback model ID for executor sessions. |
|
||||||
|
| `executionFallbackThinkingLevel` | `ThinkingLevel` | `undefined` | Executor fallback thinking override; inherits shared fallback thinking, then executor primary thinking. |
|
||||||
| `validatorProvider` | `string` | `undefined` | Provider for plan/code reviewers. |
|
| `validatorProvider` | `string` | `undefined` | Provider for plan/code reviewers. |
|
||||||
| `validatorModelId` | `string` | `undefined` | Model ID for plan/code reviewers. |
|
| `validatorModelId` | `string` | `undefined` | Model ID for plan/code reviewers. |
|
||||||
| `validatorFallbackProvider` | `string` | `undefined` | Fallback provider for reviewers; also used by reviewer UNAVAILABLE/error recovery retry before returning terminal UNAVAILABLE. |
|
| `validatorFallbackProvider` | `string` | `undefined` | Fallback provider for reviewers; also used by reviewer UNAVAILABLE/error recovery retry before returning terminal UNAVAILABLE. |
|
||||||
@@ -1014,7 +1017,9 @@ Fusion resolves task models through workflow-backed lane values first, then glob
|
|||||||
|
|
||||||
Direct-chat defaults are project-scoped and independent of task workflow lanes. Configure them in **Settings -> Project Models -> Chat**. `chatDefaultKind: "agent"` resolves only when `chatDefaultAgentId` is set; `chatDefaultKind: "model"` resolves only when both `chatDefaultModelProvider` and `chatDefaultModelId` are set, with optional `chatDefaultThinkingLevel`. If `chatNewSessionMode` is `"always-default"` and that target resolves, every New Chat entry point creates the session directly. If the target is incomplete, or the mode is unset/`"prompt"`, Fusion opens the New Chat dialog instead and preselects the resolved default when one exists. Chat Rooms additionally support a per-room `thinkingLevel` default that applies to every room responder; clearing it inherits the resolved project/global default.
|
Direct-chat defaults are project-scoped and independent of task workflow lanes. Configure them in **Settings -> Project Models -> Chat**. `chatDefaultKind: "agent"` resolves only when `chatDefaultAgentId` is set; `chatDefaultKind: "model"` resolves only when both `chatDefaultModelProvider` and `chatDefaultModelId` are set, with optional `chatDefaultThinkingLevel`. If `chatNewSessionMode` is `"always-default"` and that target resolves, every New Chat entry point creates the session directly. If the target is incomplete, or the mode is unset/`"prompt"`, Fusion opens the New Chat dialog instead and preselects the resolved default when one exists. Chat Rooms additionally support a per-room `thinkingLevel` default that applies to every room responder; clearing it inherits the resolved project/global default.
|
||||||
|
|
||||||
Settings model lanes can also carry optional thinking/reasoning effort overrides in the same model dropdown. Primary workflow lanes declare `executionThinkingLevel`, `planningThinkingLevel`, or `validatorThinkingLevel` per `(workflow, project)`; planning/reviewer fallback lanes declare `planningFallbackThinkingLevel` and `validatorFallbackThinkingLevel`; global fallback uses `fallbackThinkingLevel`; and project title summarization fallback uses `titleSummarizerFallbackThinkingLevel`. Empty thinking values inherit through the lane/global/default chain and explicit values are cleared by the lane reset action. Runtime thinking precedence for task/workflow execution is node/step `config.thinkingLevel` > lane-specific task override (`planningThinkingLevel` or `validatorThinkingLevel`) > shared task `thinkingLevel` > workflow lane thinking override > global lane thinking override > project default thinking override > global `defaultThinkingLevel`; executor sessions continue to use shared task `thinkingLevel` directly. Model-mode Chat sessions use the same executor-lane resolver with session `thinkingLevel` in the task slot, so an empty chat-session value inherits project/global defaults while a concrete New Chat selection wins for that session. The resolved value still flows through pi.ts' existing thinking/reasoning-conflict fallback (Fusion retries without the explicit level when a provider rejects conflicting thinking parameters).
|
Settings model lanes can also carry optional thinking/reasoning effort overrides in the same model dropdown. Primary workflow lanes declare `executionThinkingLevel`, `planningThinkingLevel`, or `validatorThinkingLevel` per `(workflow, project)`; executor/planning/reviewer fallback lanes declare `executionFallbackThinkingLevel`, `planningFallbackThinkingLevel`, and `validatorFallbackThinkingLevel`; global fallback uses `fallbackThinkingLevel`; and project title summarization fallback uses `titleSummarizerFallbackThinkingLevel`. Empty thinking values inherit through the lane/global/default chain and explicit values are cleared by the lane reset action. Runtime thinking precedence for task/workflow execution is node/step `config.thinkingLevel` > lane-specific task override (`planningThinkingLevel` or `validatorThinkingLevel`) > shared task `thinkingLevel` > workflow lane thinking override > global lane thinking override > project default thinking override > global `defaultThinkingLevel`; executor sessions continue to use shared task `thinkingLevel` directly. Model-mode Chat sessions use the same executor-lane resolver with session `thinkingLevel` in the task slot, so an empty chat-session value inherits project/global defaults while a concrete New Chat selection wins for that session. The resolved value still flows through pi.ts' existing thinking/reasoning-conflict fallback (Fusion retries without the explicit level when a provider rejects conflicting thinking parameters).
|
||||||
|
|
||||||
|
Executor sessions, including workflow-step timeout/malformed-output recovery and durable heartbeats, resolve `executionFallbackProvider`/`executionFallbackModelId` first and otherwise inherit the global `fallbackProvider`/`fallbackModelId` pair. For a distinct complete fallback pair, model-selection recovery is bounded to **primary → fallback → primary**. If all three attempts fail, Fusion raises an operator-actionable terminal failure with the standard retry affordance; missing, incomplete, or equal fallback pairs remain terminal after the initial primary failure.
|
||||||
|
|
||||||
When the planning lane has neither `planningFallback*` nor a global `fallback*` pair configured, triage now derives an **implicit fallback** from the resolved project/global default (execution) model (FN-7719). This lets a retryable primary planner-model failure (e.g. a provider 404/429) recover via one distinct swap instead of permanently failing triage with "no fallback configured" — the operator's chosen primary planner lane is unchanged, and the implicit fallback is skipped when it would equal the primary model or when test mode is active.
|
When the planning lane has neither `planningFallback*` nor a global `fallback*` pair configured, triage now derives an **implicit fallback** from the resolved project/global default (execution) model (FN-7719). This lets a retryable primary planner-model failure (e.g. a provider 404/429) recover via one distinct swap instead of permanently failing triage with "no fallback configured" — the operator's chosen primary planner lane is unchanged, and the implicit fallback is skipped when it would equal the primary model or when test mode is active.
|
||||||
|
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
|
|||||||
import {
|
import {
|
||||||
applyTestModeOverrides,
|
applyTestModeOverrides,
|
||||||
resolveExecutionSettingsModel,
|
resolveExecutionSettingsModel,
|
||||||
|
resolveExecutorFallbackModel,
|
||||||
resolvePlanningSettingsModel,
|
resolvePlanningSettingsModel,
|
||||||
resolveProjectDefaultModel,
|
resolveProjectDefaultModel,
|
||||||
resolveTaskExecutionModel,
|
resolveTaskExecutionModel,
|
||||||
@@ -15,6 +16,18 @@ import {
|
|||||||
} from "../model-resolution.js";
|
} from "../model-resolution.js";
|
||||||
|
|
||||||
describe("model-resolution", () => {
|
describe("model-resolution", () => {
|
||||||
|
it("resolves executor fallback before the shared fallback and forces mock in test mode", () => {
|
||||||
|
expect(resolveExecutorFallbackModel({
|
||||||
|
executionFallbackProvider: "executor-provider",
|
||||||
|
executionFallbackModelId: "executor-model",
|
||||||
|
fallbackProvider: "global-provider",
|
||||||
|
fallbackModelId: "global-model",
|
||||||
|
})).toEqual({ provider: "executor-provider", modelId: "executor-model" });
|
||||||
|
expect(resolveExecutorFallbackModel({ fallbackProvider: "global-provider", fallbackModelId: "global-model" })).toEqual({ provider: "global-provider", modelId: "global-model" });
|
||||||
|
expect(resolveExecutorFallbackModel({})).toEqual({ provider: undefined, modelId: undefined });
|
||||||
|
expect(resolveExecutorFallbackModel({ testMode: true, fallbackProvider: "global-provider", fallbackModelId: "global-model" })).toEqual(TEST_MODE_RESOLVED);
|
||||||
|
});
|
||||||
|
|
||||||
it("prefers the project default override over the global default", () => {
|
it("prefers the project default override over the global default", () => {
|
||||||
expect(
|
expect(
|
||||||
resolveProjectDefaultModel({
|
resolveProjectDefaultModel({
|
||||||
|
|||||||
@@ -317,6 +317,9 @@ describe("settings key parity", () => {
|
|||||||
"reflectionEnabled",
|
"reflectionEnabled",
|
||||||
"executionProvider",
|
"executionProvider",
|
||||||
"executionModelId",
|
"executionModelId",
|
||||||
|
"executionFallbackProvider",
|
||||||
|
"executionFallbackModelId",
|
||||||
|
"executionFallbackThinkingLevel",
|
||||||
"planningProvider",
|
"planningProvider",
|
||||||
"planningModelId",
|
"planningModelId",
|
||||||
"planningFallbackProvider",
|
"planningFallbackProvider",
|
||||||
@@ -574,6 +577,8 @@ describe("model lane key parity regression (FN-1729)", () => {
|
|||||||
{ provider: "planningProvider", modelId: "planningModelId", expectedScope: "workflow" },
|
{ provider: "planningProvider", modelId: "planningModelId", expectedScope: "workflow" },
|
||||||
{ provider: "planningGlobalProvider", modelId: "planningGlobalModelId", expectedScope: "global" },
|
{ provider: "planningGlobalProvider", modelId: "planningGlobalModelId", expectedScope: "global" },
|
||||||
{ provider: "planningFallbackProvider", modelId: "planningFallbackModelId", expectedScope: "workflow" },
|
{ provider: "planningFallbackProvider", modelId: "planningFallbackModelId", expectedScope: "workflow" },
|
||||||
|
// Executor fallback lane
|
||||||
|
{ provider: "executionFallbackProvider", modelId: "executionFallbackModelId", expectedScope: "workflow" },
|
||||||
// Validator lane
|
// Validator lane
|
||||||
{ provider: "validatorProvider", modelId: "validatorModelId", expectedScope: "workflow" },
|
{ provider: "validatorProvider", modelId: "validatorModelId", expectedScope: "workflow" },
|
||||||
{ provider: "validatorGlobalProvider", modelId: "validatorGlobalModelId", expectedScope: "global" },
|
{ provider: "validatorGlobalProvider", modelId: "validatorGlobalModelId", expectedScope: "global" },
|
||||||
|
|||||||
@@ -229,6 +229,30 @@ export const BUILTIN_MOVED_WORKFLOW_SETTINGS: WorkflowSettingDefinition[] = [
|
|||||||
options: THINKING_LEVELS.map((level) => ({ value: level, label: level })),
|
options: THINKING_LEVELS.map((level) => ({ value: level, label: level })),
|
||||||
description: "Thinking effort for the execution phase. Empty inherits from the task or default thinking level.",
|
description: "Thinking effort for the execution phase. Empty inherits from the task or default thinking level.",
|
||||||
},
|
},
|
||||||
|
/*
|
||||||
|
* FNXC:Settings-ExecutorModel 2026-07-16-00:00:
|
||||||
|
* FN-8098 makes executor recovery workflow-configurable; unset values deliberately
|
||||||
|
* inherit the shared fallback pair so existing configurations continue to work.
|
||||||
|
*/
|
||||||
|
{
|
||||||
|
id: "executionFallbackProvider",
|
||||||
|
name: "Executor fallback provider",
|
||||||
|
type: "string",
|
||||||
|
description: "Fallback provider for the execution phase.",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "executionFallbackModelId",
|
||||||
|
name: "Executor fallback model",
|
||||||
|
type: "string",
|
||||||
|
description: "Fallback model id for the execution phase.",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "executionFallbackThinkingLevel",
|
||||||
|
name: "Executor fallback thinking level",
|
||||||
|
type: "enum",
|
||||||
|
options: THINKING_LEVELS.map((level) => ({ value: level, label: level })),
|
||||||
|
description: "Thinking effort for the executor fallback model. Empty inherits from shared fallback or executor thinking.",
|
||||||
|
},
|
||||||
{
|
{
|
||||||
id: "planningProvider",
|
id: "planningProvider",
|
||||||
name: "Planning provider",
|
name: "Planning provider",
|
||||||
|
|||||||
@@ -1371,6 +1371,7 @@ export {
|
|||||||
applyTestModeOverrides,
|
applyTestModeOverrides,
|
||||||
isTestModeActive,
|
isTestModeActive,
|
||||||
resolveExecutionSettingsModel,
|
resolveExecutionSettingsModel,
|
||||||
|
resolveExecutorFallbackModel,
|
||||||
resolveMergerFallbackModel,
|
resolveMergerFallbackModel,
|
||||||
resolveMergerSettingsModel,
|
resolveMergerSettingsModel,
|
||||||
resolvePlanningSettingsModel,
|
resolvePlanningSettingsModel,
|
||||||
|
|||||||
@@ -1423,6 +1423,7 @@ export {
|
|||||||
applyTestModeOverrides,
|
applyTestModeOverrides,
|
||||||
isTestModeActive,
|
isTestModeActive,
|
||||||
resolveExecutionSettingsModel,
|
resolveExecutionSettingsModel,
|
||||||
|
resolveExecutorFallbackModel,
|
||||||
resolveMergerFallbackModel,
|
resolveMergerFallbackModel,
|
||||||
resolveMergerSettingsModel,
|
resolveMergerSettingsModel,
|
||||||
resolvePhaseThinkingLevel,
|
resolvePhaseThinkingLevel,
|
||||||
|
|||||||
@@ -262,6 +262,27 @@ export function resolveMergerFallbackModel(settings?: Partial<Settings>): Resolv
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* FNXC:Settings-ExecutorModel 2026-07-16-00:00:
|
||||||
|
* FN-8098 gives executor work a workflow-specific fallback pair while preserving the
|
||||||
|
* shared fallback as the ultimate default for workflows that leave this lane unset.
|
||||||
|
*/
|
||||||
|
export function resolveExecutorFallbackModel(settings?: Partial<Settings>): ResolvedModelSelection {
|
||||||
|
return applyTestModeOverrides(
|
||||||
|
pickFirstModelPair(
|
||||||
|
{
|
||||||
|
provider: settings?.executionFallbackProvider,
|
||||||
|
modelId: settings?.executionFallbackModelId,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
provider: settings?.fallbackProvider,
|
||||||
|
modelId: settings?.fallbackModelId,
|
||||||
|
},
|
||||||
|
),
|
||||||
|
settings,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
export function resolveTaskExecutionModel(
|
export function resolveTaskExecutionModel(
|
||||||
task: TaskModelLike,
|
task: TaskModelLike,
|
||||||
settings?: Partial<Settings>,
|
settings?: Partial<Settings>,
|
||||||
|
|||||||
@@ -43,6 +43,9 @@ type MovedProjectSettingsKey =
|
|||||||
| "executionProvider"
|
| "executionProvider"
|
||||||
| "executionModelId"
|
| "executionModelId"
|
||||||
| "executionThinkingLevel"
|
| "executionThinkingLevel"
|
||||||
|
| "executionFallbackProvider"
|
||||||
|
| "executionFallbackModelId"
|
||||||
|
| "executionFallbackThinkingLevel"
|
||||||
| "planningProvider"
|
| "planningProvider"
|
||||||
| "planningModelId"
|
| "planningModelId"
|
||||||
| "planningThinkingLevel"
|
| "planningThinkingLevel"
|
||||||
|
|||||||
@@ -3377,6 +3377,17 @@ export interface ProjectSettings {
|
|||||||
executionModelId?: string;
|
executionModelId?: string;
|
||||||
/** Workflow-declared execution-lane thinking override. Inherits through task/default thinking when unset. */
|
/** Workflow-declared execution-lane thinking override. Inherits through task/default thinking when unset. */
|
||||||
executionThinkingLevel?: ThinkingLevel;
|
executionThinkingLevel?: ThinkingLevel;
|
||||||
|
/*
|
||||||
|
* FNXC:Settings-ExecutorModel 2026-07-16-00:00:
|
||||||
|
* FN-8098 lets execution sessions select their own recovery model before the shared
|
||||||
|
* fallback pair, so reviewer, merger, planning, and executor lanes can recover independently.
|
||||||
|
*/
|
||||||
|
/** Workflow fallback provider for executor sessions. Must pair with `executionFallbackModelId`; resolves before the shared global fallback pair. */
|
||||||
|
executionFallbackProvider?: string;
|
||||||
|
/** Workflow fallback model ID for executor sessions. Must pair with `executionFallbackProvider`; resolves before the shared global fallback pair. */
|
||||||
|
executionFallbackModelId?: string;
|
||||||
|
/** Workflow executor-fallback thinking override. Inherits shared fallback thinking, then executor primary thinking. */
|
||||||
|
executionFallbackThinkingLevel?: ThinkingLevel;
|
||||||
/** Workflow-declared planning-lane thinking override. Inherits through task/default thinking when unset. */
|
/** Workflow-declared planning-lane thinking override. Inherits through task/default thinking when unset. */
|
||||||
planningThinkingLevel?: ThinkingLevel;
|
planningThinkingLevel?: ThinkingLevel;
|
||||||
/** AI model provider for validator/reviewer agent.
|
/** AI model provider for validator/reviewer agent.
|
||||||
|
|||||||
@@ -541,6 +541,14 @@ export const WORKFLOW_MODEL_LANE_CATALOG: WorkflowModelLanePair[] = [
|
|||||||
label: "Reviewer Model",
|
label: "Reviewer Model",
|
||||||
help: "Provider and model used by review and validation agents. Leave unset to inherit from the default lane.",
|
help: "Provider and model used by review and validation agents. Leave unset to inherit from the default lane.",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
id: "execution-fallback",
|
||||||
|
providerId: "executionFallbackProvider",
|
||||||
|
modelId: "executionFallbackModelId",
|
||||||
|
thinkingId: "executionFallbackThinkingLevel",
|
||||||
|
label: "Executor Fallback Model",
|
||||||
|
help: "Fallback provider and model used when the primary Executor model cannot be used.",
|
||||||
|
},
|
||||||
{
|
{
|
||||||
id: "planning-fallback",
|
id: "planning-fallback",
|
||||||
providerId: "planningFallbackProvider",
|
providerId: "planningFallbackProvider",
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import { LoadingSpinner } from "../../LoadingSpinner";
|
|||||||
import { useAgentsMapCache } from "../../../hooks/useAgentsMapCache";
|
import { useAgentsMapCache } from "../../../hooks/useAgentsMapCache";
|
||||||
type LaneStatus = "inherited" | "overridden";
|
type LaneStatus = "inherited" | "overridden";
|
||||||
type WorkflowModelPair = {
|
type WorkflowModelPair = {
|
||||||
id: "planning" | "execution" | "validator" | "planning-fallback" | "validator-fallback";
|
id: "planning" | "execution" | "validator" | "execution-fallback" | "planning-fallback" | "validator-fallback";
|
||||||
providerId: string;
|
providerId: string;
|
||||||
modelId: string;
|
modelId: string;
|
||||||
thinkingId?: string;
|
thinkingId?: string;
|
||||||
@@ -55,6 +55,14 @@ const WORKFLOW_MODEL_PAIRS: WorkflowModelPair[] = [
|
|||||||
label: "Reviewer Model",
|
label: "Reviewer Model",
|
||||||
help: "Provider and model used for workflow review or validation lanes. Leave unset to inherit from the workflow default.",
|
help: "Provider and model used for workflow review or validation lanes. Leave unset to inherit from the workflow default.",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
id: "execution-fallback",
|
||||||
|
providerId: "executionFallbackProvider",
|
||||||
|
modelId: "executionFallbackModelId",
|
||||||
|
thinkingId: "executionFallbackThinkingLevel",
|
||||||
|
label: "Executor Fallback Model",
|
||||||
|
help: "Fallback provider and model used when the primary Executor model cannot be used.",
|
||||||
|
},
|
||||||
{
|
{
|
||||||
id: "planning-fallback",
|
id: "planning-fallback",
|
||||||
providerId: "planningFallbackProvider",
|
providerId: "planningFallbackProvider",
|
||||||
|
|||||||
@@ -1176,9 +1176,15 @@ describe("piLog structured diagnostics", () => {
|
|||||||
} as unknown as AgentSession;
|
} as unknown as AgentSession;
|
||||||
|
|
||||||
createAgentSessionMock.mockReset();
|
createAgentSessionMock.mockReset();
|
||||||
|
const primaryRetrySession = {
|
||||||
|
model: { provider: "openai", id: "gpt-4o" },
|
||||||
|
prompt: vi.fn().mockRejectedValue(new Error("429 Too Many Requests")),
|
||||||
|
subscribe: vi.fn(), dispose: vi.fn(), setThinkingLevel: vi.fn(), sessionFile: undefined,
|
||||||
|
} as unknown as AgentSession;
|
||||||
createAgentSessionMock
|
createAgentSessionMock
|
||||||
.mockResolvedValueOnce({ session: primarySession } as any)
|
.mockResolvedValueOnce({ session: primarySession } as any)
|
||||||
.mockResolvedValueOnce({ session: fallbackSession } as any);
|
.mockResolvedValueOnce({ session: fallbackSession } as any)
|
||||||
|
.mockResolvedValueOnce({ session: primaryRetrySession } as any);
|
||||||
|
|
||||||
const { session } = await createFnAgent({
|
const { session } = await createFnAgent({
|
||||||
cwd: "/test/project",
|
cwd: "/test/project",
|
||||||
@@ -1193,15 +1199,17 @@ describe("piLog structured diagnostics", () => {
|
|||||||
|
|
||||||
await expect((session as any).promptWithFallback("prompt text")).rejects.toMatchObject({
|
await expect((session as any).promptWithFallback("prompt text")).rejects.toMatchObject({
|
||||||
name: "ModelFallbackExhaustedError",
|
name: "ModelFallbackExhaustedError",
|
||||||
attempts: 2,
|
attempts: 3,
|
||||||
primaryModel: "openai/gpt-4o",
|
primaryModel: "openai/gpt-4o",
|
||||||
fallbackModel: "anthropic/claude-3-5-haiku-20241022",
|
fallbackModel: "anthropic/claude-3-5-haiku-20241022",
|
||||||
triggerPoint: "prompt-time",
|
triggerPoint: "prompt-time",
|
||||||
});
|
});
|
||||||
|
|
||||||
expect(createAgentSessionMock).toHaveBeenCalledTimes(2);
|
expect(createAgentSessionMock).toHaveBeenCalledTimes(3);
|
||||||
expect(primarySession.prompt).toHaveBeenCalledTimes(1);
|
expect(primarySession.prompt).toHaveBeenCalledTimes(1);
|
||||||
expect(fallbackSession.prompt).toHaveBeenCalledTimes(1);
|
expect(fallbackSession.prompt).toHaveBeenCalledTimes(1);
|
||||||
|
expect(primaryRetrySession.prompt).toHaveBeenCalledTimes(1);
|
||||||
|
expect((createAgentSessionMock.mock.calls[2]?.[0] as any).model.id).toBe("gpt-4o");
|
||||||
expect(onFallbackModelUsed).toHaveBeenCalledTimes(1);
|
expect(onFallbackModelUsed).toHaveBeenCalledTimes(1);
|
||||||
expect(onFallbackModelUsed).toHaveBeenCalledWith(expect.objectContaining({
|
expect(onFallbackModelUsed).toHaveBeenCalledWith(expect.objectContaining({
|
||||||
triggerPoint: "prompt-time",
|
triggerPoint: "prompt-time",
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ import {
|
|||||||
isGrokApiKeyFusionVisible,
|
isGrokApiKeyFusionVisible,
|
||||||
isTestModeActive,
|
isTestModeActive,
|
||||||
resolveExecutionSettingsModel,
|
resolveExecutionSettingsModel,
|
||||||
|
resolveExecutorFallbackModel,
|
||||||
resolveMergerSettingsModel,
|
resolveMergerSettingsModel,
|
||||||
resolvePhaseThinkingLevel,
|
resolvePhaseThinkingLevel,
|
||||||
resolveProjectDefaultModel,
|
resolveProjectDefaultModel,
|
||||||
@@ -285,6 +286,7 @@ export function resolveExecutorFallbackThinkingLevel(
|
|||||||
settings: Partial<Settings> | undefined,
|
settings: Partial<Settings> | undefined,
|
||||||
): string | undefined {
|
): string | undefined {
|
||||||
return firstThinkingLevel(
|
return firstThinkingLevel(
|
||||||
|
settings?.executionFallbackThinkingLevel,
|
||||||
settings?.fallbackThinkingLevel,
|
settings?.fallbackThinkingLevel,
|
||||||
resolveExecutorThinkingLevel(taskThinkingLevel, settings),
|
resolveExecutorThinkingLevel(taskThinkingLevel, settings),
|
||||||
);
|
);
|
||||||
@@ -567,6 +569,7 @@ export function resolveHeartbeatSessionModels(
|
|||||||
}
|
}
|
||||||
|
|
||||||
const executionSettingsModel = resolveExecutionSettingsModel(settings);
|
const executionSettingsModel = resolveExecutionSettingsModel(settings);
|
||||||
|
const executorFallbackModel = resolveExecutorFallbackModel(settings);
|
||||||
const assignedRuntimeModel = extractRuntimeModel(assignedAgentRuntimeConfig);
|
const assignedRuntimeModel = extractRuntimeModel(assignedAgentRuntimeConfig);
|
||||||
/*
|
/*
|
||||||
FNXC:AgentHeartbeat 2026-07-14-16:13:
|
FNXC:AgentHeartbeat 2026-07-14-16:13:
|
||||||
@@ -579,8 +582,8 @@ export function resolveHeartbeatSessionModels(
|
|||||||
return {
|
return {
|
||||||
defaultProvider: resolvedModel.provider,
|
defaultProvider: resolvedModel.provider,
|
||||||
defaultModelId: resolvedModel.modelId,
|
defaultModelId: resolvedModel.modelId,
|
||||||
fallbackProvider: undefined,
|
fallbackProvider: executorFallbackModel.provider,
|
||||||
fallbackModelId: undefined,
|
fallbackModelId: executorFallbackModel.modelId,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ import { existsSync, lstatSync, realpathSync } from "node:fs";
|
|||||||
import { readFile, rm, writeFile } from "node:fs/promises";
|
import { readFile, rm, writeFile } from "node:fs/promises";
|
||||||
import type { TaskStore, Task, TaskDetail, TaskTokenUsage, StepStatus, Settings, WorkflowStep, MissionStore, AsyncMissionStore, Slice, AgentState, AgentCapability, RunMutationContext, AgentHeartbeatConfig, Agent, AgentMemoryInclusionMode, ProjectSettings, MergeResult, WorkflowIrNode, WorkflowIrNodeKind, WorkflowStepResult as CoreWorkflowStepResult, ThinkingLevel } from "@fusion/core";
|
import type { TaskStore, Task, TaskDetail, TaskTokenUsage, StepStatus, Settings, WorkflowStep, MissionStore, AsyncMissionStore, Slice, AgentState, AgentCapability, RunMutationContext, AgentHeartbeatConfig, Agent, AgentMemoryInclusionMode, ProjectSettings, MergeResult, WorkflowIrNode, WorkflowIrNodeKind, WorkflowStepResult as CoreWorkflowStepResult, ThinkingLevel } from "@fusion/core";
|
||||||
import { getUnmetSchedulingDependencies } from "./scheduler.js";
|
import { getUnmetSchedulingDependencies } from "./scheduler.js";
|
||||||
import { RetryStormError, serializeRetryStormError, isExperimentalFeatureEnabled, resolveWorkflowIrForTask, resolveColumnAgentBinding, resolveEffectiveAgent, instanceNodeId, getWorkflowExtensionRegistry, getBuiltinWorkflow, parseNoOpCompletionMarker, allowsAutoMergeProcessing, resolveEffectiveAutoMerge, isLiveSharedBranchGroupMemberIntegration, resolveMaxAutoMergeRetries, resolveOptionalStepRevisionBudget, resolveOptionalReviewRevisionBudget, COMPLETION_SUMMARY_NODE_ID, upsertWorkflowStepResult, AWAITING_APPROVAL_PAUSE_REASON, THINKING_LEVELS, AgentStore } from "@fusion/core";
|
import { RetryStormError, serializeRetryStormError, isExperimentalFeatureEnabled, resolveWorkflowIrForTask, resolveColumnAgentBinding, resolveEffectiveAgent, instanceNodeId, getWorkflowExtensionRegistry, getBuiltinWorkflow, parseNoOpCompletionMarker, allowsAutoMergeProcessing, resolveEffectiveAutoMerge, isLiveSharedBranchGroupMemberIntegration, resolveMaxAutoMergeRetries, resolveOptionalStepRevisionBudget, resolveOptionalReviewRevisionBudget, COMPLETION_SUMMARY_NODE_ID, upsertWorkflowStepResult, AWAITING_APPROVAL_PAUSE_REASON, THINKING_LEVELS, AgentStore, resolveExecutorFallbackModel } from "@fusion/core";
|
||||||
import { finalizeProvenAutoMergeTask } from "./auto-merge-finalization.js";
|
import { finalizeProvenAutoMergeTask } from "./auto-merge-finalization.js";
|
||||||
import { mergeEffectiveSettings } from "./effective-settings.js";
|
import { mergeEffectiveSettings } from "./effective-settings.js";
|
||||||
import { moveTaskToReplanColumn, resolveReplanTargetColumn } from "./replan-target.js";
|
import { moveTaskToReplanColumn, resolveReplanTargetColumn } from "./replan-target.js";
|
||||||
@@ -11242,8 +11242,7 @@ export class TaskExecutor {
|
|||||||
settings,
|
settings,
|
||||||
(identityAgent?.runtimeConfig ?? undefined) as Record<string, unknown> | undefined,
|
(identityAgent?.runtimeConfig ?? undefined) as Record<string, unknown> | undefined,
|
||||||
);
|
);
|
||||||
const executorFallbackProvider = settings.fallbackProvider;
|
const { provider: executorFallbackProvider, modelId: executorFallbackModelId } = resolveExecutorFallbackModel(settings);
|
||||||
const executorFallbackModelId = settings.fallbackModelId;
|
|
||||||
const executorSessionThinkingSource = this.graphSeamThinkingLevel.get(task.id) ?? detail.thinkingLevel;
|
const executorSessionThinkingSource = this.graphSeamThinkingLevel.get(task.id) ?? detail.thinkingLevel;
|
||||||
const executorThinkingLevel = resolveExecutorThinkingLevel(executorSessionThinkingSource, settings);
|
const executorThinkingLevel = resolveExecutorThinkingLevel(executorSessionThinkingSource, settings);
|
||||||
const executorFallbackThinkingLevel = resolveExecutorFallbackThinkingLevel(executorSessionThinkingSource, settings);
|
const executorFallbackThinkingLevel = resolveExecutorFallbackThinkingLevel(executorSessionThinkingSource, settings);
|
||||||
@@ -14947,6 +14946,8 @@ export class TaskExecutor {
|
|||||||
assignedRuntimeConfig,
|
assignedRuntimeConfig,
|
||||||
);
|
);
|
||||||
|
|
||||||
|
const executorFallback = resolveExecutorFallbackModel(settings);
|
||||||
|
|
||||||
// Create the fix agent session
|
// Create the fix agent session
|
||||||
const { session } = await createResolvedAgentSession({
|
const { session } = await createResolvedAgentSession({
|
||||||
sessionPurpose: "executor",
|
sessionPurpose: "executor",
|
||||||
@@ -14975,6 +14976,9 @@ Do not refactor, rename broadly, or make opportunistic improvements.
|
|||||||
onToolEnd: logger.onToolEnd,
|
onToolEnd: logger.onToolEnd,
|
||||||
defaultProvider: executorProvider,
|
defaultProvider: executorProvider,
|
||||||
defaultModelId: executorModelId,
|
defaultModelId: executorModelId,
|
||||||
|
fallbackProvider: executorFallback.provider,
|
||||||
|
fallbackModelId: executorFallback.modelId,
|
||||||
|
fallbackThinkingLevel: resolveExecutorFallbackThinkingLevel(task.thinkingLevel, settings),
|
||||||
defaultThinkingLevel: resolveExecutorThinkingLevel(task.thinkingLevel, settings),
|
defaultThinkingLevel: resolveExecutorThinkingLevel(task.thinkingLevel, settings),
|
||||||
runAuditor: createRunAuditor(this.store, this.getRunContextFor(task.id)),
|
runAuditor: createRunAuditor(this.store, this.getRunContextFor(task.id)),
|
||||||
settings,
|
settings,
|
||||||
@@ -15895,8 +15899,8 @@ You have access to the file system to review changes.${inlineFixBlock}${verdictB
|
|||||||
// own override takes precedence; otherwise use the canonical executor
|
// own override takes precedence; otherwise use the canonical executor
|
||||||
// hierarchy: task override → project execution lane → global execution lane
|
// hierarchy: task override → project execution lane → global execution lane
|
||||||
// → project default override → global default. The fallback is the per-step
|
// → project default override → global default. The fallback is the per-step
|
||||||
// override's missing-counterpart settings, then the global validator/fallback
|
// override's missing-counterpart settings, then the executor fallback lane,
|
||||||
// pair, then the executor's `fallbackProvider`.
|
// which itself falls through to the shared global fallback pair.
|
||||||
// FNXC:ModelResolution 2026-06-25-12:00: FN-7039 requires workflow steps to inherit project execution-lane model settings before default settings so configured Execution models reach step sessions unless the step itself overrides them.
|
// FNXC:ModelResolution 2026-06-25-12:00: FN-7039 requires workflow steps to inherit project execution-lane model settings before default settings so configured Execution models reach step sessions unless the step itself overrides them.
|
||||||
const assignedRuntimeConfig = await this.getAssignedAgentRuntimeConfig(task.assignedAgentId);
|
const assignedRuntimeConfig = await this.getAssignedAgentRuntimeConfig(task.assignedAgentId);
|
||||||
const executorModel = resolveExecutorSessionModel(
|
const executorModel = resolveExecutorSessionModel(
|
||||||
@@ -15909,15 +15913,11 @@ You have access to the file system to review changes.${inlineFixBlock}${verdictB
|
|||||||
const primaryModelId = workflowStep.modelId || executorModel.modelId;
|
const primaryModelId = workflowStep.modelId || executorModel.modelId;
|
||||||
const useOverride = !!(workflowStep.modelProvider && workflowStep.modelId);
|
const useOverride = !!(workflowStep.modelProvider && workflowStep.modelId);
|
||||||
|
|
||||||
type ModelTuple = { provider?: string; modelId?: string };
|
const executorFallback = resolveExecutorFallbackModel(settings);
|
||||||
type WorkflowStepFallbackLabel = "validatorFallback" | "globalFallback";
|
const fallback = executorFallback.provider && executorFallback.modelId
|
||||||
const fallbackCandidates: Array<ModelTuple & { label: WorkflowStepFallbackLabel }> = [
|
&& (executorFallback.provider !== primaryProvider || executorFallback.modelId !== primaryModelId)
|
||||||
{ provider: settings.validatorFallbackProvider, modelId: settings.validatorFallbackModelId, label: "validatorFallback" },
|
? executorFallback
|
||||||
{ provider: settings.fallbackProvider, modelId: settings.fallbackModelId, label: "globalFallback" },
|
: undefined;
|
||||||
];
|
|
||||||
const fallback = fallbackCandidates.find(
|
|
||||||
(c) => c.provider && c.modelId && (c.provider !== primaryProvider || c.modelId !== primaryModelId),
|
|
||||||
);
|
|
||||||
|
|
||||||
const timeoutMs = Math.max(60_000, settings.workflowStepTimeoutMs ?? 900_000);
|
const timeoutMs = Math.max(60_000, settings.workflowStepTimeoutMs ?? 900_000);
|
||||||
|
|
||||||
@@ -16064,9 +16064,7 @@ You have access to the file system to review changes.${inlineFixBlock}${verdictB
|
|||||||
*/
|
*/
|
||||||
const workflowStepThinkingSource = workflowStep.thinkingLevel ?? task.thinkingLevel;
|
const workflowStepThinkingSource = workflowStep.thinkingLevel ?? task.thinkingLevel;
|
||||||
const workflowStepThinkingLevel = attemptLabel === "fallback"
|
const workflowStepThinkingLevel = attemptLabel === "fallback"
|
||||||
? (fallback?.label === "validatorFallback"
|
? resolveExecutorFallbackThinkingLevel(workflowStepThinkingSource, settings)
|
||||||
? resolveValidatorFallbackThinkingLevel(workflowStepThinkingSource, settings)
|
|
||||||
: resolveExecutorFallbackThinkingLevel(workflowStepThinkingSource, settings))
|
|
||||||
: resolveExecutorThinkingLevel(workflowStepThinkingSource, settings);
|
: resolveExecutorThinkingLevel(workflowStepThinkingSource, settings);
|
||||||
const workflowStepFallbackThinkingLevel = resolveExecutorFallbackThinkingLevel(workflowStepThinkingSource, settings);
|
const workflowStepFallbackThinkingLevel = resolveExecutorFallbackThinkingLevel(workflowStepThinkingSource, settings);
|
||||||
const { session } = await createResolvedAgentSession({
|
const { session } = await createResolvedAgentSession({
|
||||||
@@ -16078,8 +16076,8 @@ You have access to the file system to review changes.${inlineFixBlock}${verdictB
|
|||||||
tools: toolMode,
|
tools: toolMode,
|
||||||
defaultProvider: provider,
|
defaultProvider: provider,
|
||||||
defaultModelId: modelId,
|
defaultModelId: modelId,
|
||||||
fallbackProvider: settings.fallbackProvider,
|
fallbackProvider: executorFallback.provider,
|
||||||
fallbackModelId: settings.fallbackModelId,
|
fallbackModelId: executorFallback.modelId,
|
||||||
fallbackThinkingLevel: workflowStepFallbackThinkingLevel,
|
fallbackThinkingLevel: workflowStepFallbackThinkingLevel,
|
||||||
defaultThinkingLevel: workflowStepThinkingLevel,
|
defaultThinkingLevel: workflowStepThinkingLevel,
|
||||||
runAuditor: createRunAuditor(this.store, this.getRunContextFor(task.id)),
|
runAuditor: createRunAuditor(this.store, this.getRunContextFor(task.id)),
|
||||||
@@ -16272,7 +16270,7 @@ You have access to the file system to review changes.${inlineFixBlock}${verdictB
|
|||||||
if (!retryMalformed) return retryOutcome;
|
if (!retryMalformed) return retryOutcome;
|
||||||
await this.store.logEntry(
|
await this.store.logEntry(
|
||||||
task.id,
|
task.id,
|
||||||
`Workflow step '${workflowStep.name}' produced malformed output on both the primary attempt and one self-retry — no fallback model configured (set settings.validatorFallbackProvider/Id or fallbackProvider/Id)`,
|
`Workflow step '${workflowStep.name}' produced malformed output on both the primary attempt and one self-retry — no fallback model configured (set settings.executionFallbackProvider/Id or fallbackProvider/Id)`,
|
||||||
);
|
);
|
||||||
return retryOutcome;
|
return retryOutcome;
|
||||||
}
|
}
|
||||||
@@ -16280,12 +16278,12 @@ You have access to the file system to review changes.${inlineFixBlock}${verdictB
|
|||||||
executorLog.warn(`${task.id}: workflow step '${workflowStep.name}' ${reason} and no fallback model is configured`);
|
executorLog.warn(`${task.id}: workflow step '${workflowStep.name}' ${reason} and no fallback model is configured`);
|
||||||
await this.store.logEntry(
|
await this.store.logEntry(
|
||||||
task.id,
|
task.id,
|
||||||
`Workflow step '${workflowStep.name}' ${reason} — no fallback model configured (set settings.validatorFallbackProvider/Id or fallbackProvider/Id)`,
|
`Workflow step '${workflowStep.name}' ${reason} — no fallback model configured (set settings.executionFallbackProvider/Id or fallbackProvider/Id)`,
|
||||||
);
|
);
|
||||||
return primaryOutcome;
|
return primaryOutcome;
|
||||||
}
|
}
|
||||||
|
|
||||||
executorLog.log(`${task.id}: retrying workflow step '${workflowStep.name}' with fallback ${fallback.provider}/${fallback.modelId} (label=${fallback.label}) after primary ${primaryOutcome.timedOut ? "timeout" : "malformed output"}`);
|
executorLog.log(`${task.id}: retrying workflow step '${workflowStep.name}' with executor fallback ${fallback.provider}/${fallback.modelId} after primary ${primaryOutcome.timedOut ? "timeout" : "malformed output"}`);
|
||||||
return runOnce(fallback.provider, fallback.modelId, "fallback");
|
return runOnce(fallback.provider, fallback.modelId, "fallback");
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -18945,6 +18943,8 @@ Child agent: ${agent.id} (${name})`;
|
|||||||
const { provider: childExecutorProvider, modelId: childExecutorModelId } =
|
const { provider: childExecutorProvider, modelId: childExecutorModelId } =
|
||||||
resolveExecutorSessionModel(undefined, undefined, settings, agent.runtimeConfig as Record<string, unknown> | undefined);
|
resolveExecutorSessionModel(undefined, undefined, settings, agent.runtimeConfig as Record<string, unknown> | undefined);
|
||||||
|
|
||||||
|
const childExecutorFallback = resolveExecutorFallbackModel(settings);
|
||||||
|
|
||||||
// Create child agent session
|
// Create child agent session
|
||||||
const { session: childSession } = await createResolvedAgentSession({
|
const { session: childSession } = await createResolvedAgentSession({
|
||||||
sessionPurpose: "executor",
|
sessionPurpose: "executor",
|
||||||
@@ -18955,8 +18955,8 @@ Child agent: ${agent.id} (${name})`;
|
|||||||
tools: "coding",
|
tools: "coding",
|
||||||
defaultProvider: childExecutorProvider,
|
defaultProvider: childExecutorProvider,
|
||||||
defaultModelId: childExecutorModelId,
|
defaultModelId: childExecutorModelId,
|
||||||
fallbackProvider: settings.fallbackProvider,
|
fallbackProvider: childExecutorFallback.provider,
|
||||||
fallbackModelId: settings.fallbackModelId,
|
fallbackModelId: childExecutorFallback.modelId,
|
||||||
fallbackThinkingLevel: resolveExecutorFallbackThinkingLevel(undefined, settings),
|
fallbackThinkingLevel: resolveExecutorFallbackThinkingLevel(undefined, settings),
|
||||||
runAuditor: createRunAuditor(this.store, this.getRunContextFor(taskId)),
|
runAuditor: createRunAuditor(this.store, this.getRunContextFor(taskId)),
|
||||||
settings,
|
settings,
|
||||||
|
|||||||
@@ -2555,8 +2555,10 @@ export async function createFnAgent(options: AgentOptions): Promise<AgentResult>
|
|||||||
};
|
};
|
||||||
|
|
||||||
/*
|
/*
|
||||||
* FNXC:ModelFallback 2026-07-02-00:00:
|
* FNXC:ModelFallback 2026-07-16-00:00:
|
||||||
* Planner and shared AI lanes may try one distinct fallback model for a logical model-selection failure, but they must then throw ModelFallbackExhaustedError instead of swapping back or relying on scheduler re-pick loops. This keeps transient fallback useful while making exhausted model configuration actionable for operators.
|
* FN-8098 requires a distinct fallback failure to get one final primary retry before
|
||||||
|
* blocking. This bounded primary → fallback → primary sequence prevents fallback loops
|
||||||
|
* while still surfacing an operator-actionable ModelFallbackExhaustedError after three attempts.
|
||||||
*/
|
*/
|
||||||
let sessionResult;
|
let sessionResult;
|
||||||
let usingFallback = false;
|
let usingFallback = false;
|
||||||
@@ -2572,8 +2574,14 @@ export async function createFnAgent(options: AgentOptions): Promise<AgentResult>
|
|||||||
usingFallback = true;
|
usingFallback = true;
|
||||||
try {
|
try {
|
||||||
sessionResult = await createSessionWithModel(fallbackModel);
|
sessionResult = await createSessionWithModel(fallbackModel);
|
||||||
} catch (fallbackErr: unknown) {
|
} catch (_fallbackErr: unknown) {
|
||||||
throw makeFallbackExhaustedError("session-creation", 2, fallbackErr);
|
// The final retry is intentionally primary and terminal even when fallback failed non-retryably.
|
||||||
|
usingFallback = false;
|
||||||
|
try {
|
||||||
|
sessionResult = await createSessionWithModel(selectedModel);
|
||||||
|
} catch (primaryRetryErr: unknown) {
|
||||||
|
throw makeFallbackExhaustedError("session-creation", 3, primaryRetryErr);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
await emitFallbackUsed("session-creation", err);
|
await emitFallbackUsed("session-creation", err);
|
||||||
piLog.log("Fallback session created successfully");
|
piLog.log("Fallback session created successfully");
|
||||||
@@ -2736,50 +2744,21 @@ export async function createFnAgent(options: AgentOptions): Promise<AgentResult>
|
|||||||
try {
|
try {
|
||||||
await promptSessionAndCheck(fallbackSession, prompt, effectivePromptOptions);
|
await promptSessionAndCheck(fallbackSession, prompt, effectivePromptOptions);
|
||||||
return;
|
return;
|
||||||
} catch (fallbackErr: any) {
|
} catch (_fallbackErr: unknown) {
|
||||||
const fallbackErrorMessage = fallbackErr?.message || "";
|
/*
|
||||||
if (isContextLimitError(fallbackErrorMessage)) {
|
* FNXC:ModelFallback 2026-07-16-00:00:
|
||||||
const promptMemoryRetry = await retryWithCompactedPromptMemory(fallbackSession, prompt, effectivePromptOptions);
|
* Once a distinct fallback has failed, retry the primary exactly once regardless
|
||||||
if (promptMemoryRetry.recovered) {
|
* of whether the fallback error is retryable or a context/compaction failure.
|
||||||
return;
|
* Resetting `usingFallback` ensures the final primary receives primary thinking.
|
||||||
}
|
*/
|
||||||
if (promptMemoryRetry.error) {
|
usingFallback = false;
|
||||||
const retryMessage = promptMemoryRetry.error instanceof Error ? promptMemoryRetry.error.message : String(promptMemoryRetry.error);
|
try {
|
||||||
if (!isContextLimitError(retryMessage)) {
|
const primaryRetrySession = await swapPromptSession(selectedModel);
|
||||||
throw promptMemoryRetry.error;
|
await promptSessionAndCheck(primaryRetrySession, prompt, effectivePromptOptions);
|
||||||
}
|
return;
|
||||||
}
|
} catch (primaryRetryErr: unknown) {
|
||||||
|
throw makeFallbackExhaustedError("prompt-time", 3, primaryRetryErr);
|
||||||
const promptSectionRetry = await retryWithCompactedPromptSections(fallbackSession, prompt, effectivePromptOptions);
|
|
||||||
if (promptSectionRetry.recovered) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (promptSectionRetry.error) {
|
|
||||||
const retryMessage = promptSectionRetry.error instanceof Error ? promptSectionRetry.error.message : String(promptSectionRetry.error);
|
|
||||||
if (!isContextLimitError(retryMessage)) {
|
|
||||||
throw promptSectionRetry.error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
piLog.warn("promptWithFallback: fallback session context limit error — attempting auto-compaction");
|
|
||||||
await flushMemoryBeforeSessionCompaction(fallbackSession);
|
|
||||||
const compactResult = await compactSessionContext(fallbackSession);
|
|
||||||
if (compactResult) {
|
|
||||||
piLog.log(`promptWithFallback: fallback compaction succeeded (${compactResult.tokensBefore} tokens) — retrying`);
|
|
||||||
try {
|
|
||||||
await promptSessionAndCheck(fallbackSession, prompt, effectivePromptOptions);
|
|
||||||
return;
|
|
||||||
} catch (retryErr: any) {
|
|
||||||
const retryErrorMessage = retryErr?.message || "";
|
|
||||||
piLog.error(`promptWithFallback: fallback retry after auto-compaction failed: ${retryErrorMessage}`);
|
|
||||||
throw fallbackErr; // Throw original fallback error
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
piLog.error("promptWithFallback: fallback compaction unavailable — propagating original error");
|
|
||||||
throw fallbackErr;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
throw makeFallbackExhaustedError("prompt-time", 2, fallbackErr);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ import { existsSync } from "node:fs";
|
|||||||
import { rm } from "node:fs/promises";
|
import { rm } from "node:fs/promises";
|
||||||
import type { AgentSession } from "@earendil-works/pi-coding-agent";
|
import type { AgentSession } from "@earendil-works/pi-coding-agent";
|
||||||
import type { AgentHeartbeatRun, AgentStore, MessageStore, PermanentAgentGatingContext, ResolvedMcpServerDefinition, TaskDetail, Settings, SteeringComment, TaskStore } from "@fusion/core";
|
import type { AgentHeartbeatRun, AgentStore, MessageStore, PermanentAgentGatingContext, ResolvedMcpServerDefinition, TaskDetail, Settings, SteeringComment, TaskStore } from "@fusion/core";
|
||||||
import { resolvePersistAgentThinkingLog } from "@fusion/core";
|
import { resolvePersistAgentThinkingLog, resolveExecutorFallbackModel } from "@fusion/core";
|
||||||
|
|
||||||
import {
|
import {
|
||||||
createResolvedAgentSession,
|
createResolvedAgentSession,
|
||||||
@@ -27,6 +27,7 @@ import {
|
|||||||
promptWithAutoRetry,
|
promptWithAutoRetry,
|
||||||
resolveExecutorSessionModel,
|
resolveExecutorSessionModel,
|
||||||
resolveExecutorThinkingLevel,
|
resolveExecutorThinkingLevel,
|
||||||
|
resolveExecutorFallbackThinkingLevel,
|
||||||
} from "./agent-session-helpers.js";
|
} from "./agent-session-helpers.js";
|
||||||
import type { AgentActionGateContext } from "./agent-action-gate.js";
|
import type { AgentActionGateContext } from "./agent-action-gate.js";
|
||||||
import type { SkillSelectionContext } from "./skill-resolver.js";
|
import type { SkillSelectionContext } from "./skill-resolver.js";
|
||||||
@@ -1359,8 +1360,9 @@ Your role:
|
|||||||
Follow instructions precisely and avoid unrelated changes.`,
|
Follow instructions precisely and avoid unrelated changes.`,
|
||||||
defaultProvider: executorProvider,
|
defaultProvider: executorProvider,
|
||||||
defaultModelId: executorModelId,
|
defaultModelId: executorModelId,
|
||||||
fallbackProvider: settings.fallbackProvider,
|
fallbackProvider: resolveExecutorFallbackModel(settings).provider,
|
||||||
fallbackModelId: settings.fallbackModelId,
|
fallbackModelId: resolveExecutorFallbackModel(settings).modelId,
|
||||||
|
fallbackThinkingLevel: resolveExecutorFallbackThinkingLevel(taskDetail.thinkingLevel, settings),
|
||||||
defaultThinkingLevel: effectiveThinkingLevel,
|
defaultThinkingLevel: effectiveThinkingLevel,
|
||||||
runAuditor: createRunAuditor(this.store, {
|
runAuditor: createRunAuditor(this.store, {
|
||||||
runId: generateSyntheticRunId("workflow-step", taskDetail.id),
|
runId: generateSyntheticRunId("workflow-step", taskDetail.id),
|
||||||
|
|||||||
Reference in New Issue
Block a user