2b8df56cb8
A rate-limited reviewer filled a task's Chat tab with 14 identical
"Reviewer using model: ..." markers and no review text, hammering an
already-limited provider.
Root cause: the reviewer was the only AI lane that never classified
provider errors, so a 429 became an UNAVAILABLE verdict. With no
validator fallback configured the fallback ladder re-ran the SAME model
instantly, and fn_review_step answered with "code review remains
blocking; retry once" — bounding the loop with prompt text rather than
code. The tool's catch-all also swallowed the error into tool output, so
withRateLimitRetry, UsageLimitPauser and RetryStormError never fired.
- reviewer: throw ReviewerProviderError for usage-limit/transient errors
instead of laundering them into UNAVAILABLE, and never spend the
fallback budget (which bounds bad reviews) on an outage.
- reviewer: absorb flaky-network blips in-lane via withRetry with
jittered backoff; rate limits still escalate immediately.
- executor: re-raise the fatal after the prompt via
throwDeferredReviewerFatal — pi-agent-core converts tool throws into
tool_error results, so a tool cannot throw out of session.prompt().
- executor: give code review a real MAX_CODE_REVIEW_UNAVAILABLE_RETRIES
counter, mirroring the plan/spec limiter.
- reviewer: dedupe the model marker on text, so same-model retries stay
silent while a genuine model switch still emits.
Also fixes the run-on rendering: AgentLogType gains `status` for complete
engine messages. `text` means "streamed delta" and is re-glued with
join(""), which is why N standalone markers rendered as one string. The
split is at the type, not a separator — a separator would reintroduce the
FN-5787/5789/5803 streamed-spacing regression.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
69 lines
2.5 KiB
TypeScript
69 lines
2.5 KiB
TypeScript
import type { AgentLogType } from "@fusion/core";
|
|
import { notifyFallbackUsed } from "./notifier.js";
|
|
import type { FallbackModelUsedPayload } from "./pi.js";
|
|
|
|
type FallbackLogStore = {
|
|
logEntry?(taskId: string, action: string): Promise<unknown>;
|
|
appendAgentLog?(
|
|
taskId: string,
|
|
text: string,
|
|
// FNXC:AgentLog-EntryTypes 2026-07-15-11:20: reference the canonical AgentLogType rather than
|
|
// re-listing the members — the hand-copied union silently drifted when `status` was added.
|
|
type: AgentLogType,
|
|
detail?: string,
|
|
agent?: string,
|
|
): Promise<unknown>;
|
|
};
|
|
|
|
type FallbackModelObserverOptions = {
|
|
agent: string;
|
|
label: string;
|
|
store?: FallbackLogStore;
|
|
taskId?: string;
|
|
taskTitle?: string;
|
|
};
|
|
|
|
function buildFallbackLogMessage(
|
|
label: string,
|
|
payload: FallbackModelUsedPayload,
|
|
): string {
|
|
const reason = payload.failureCategory === "authentication"
|
|
? "; primary provider authentication failed"
|
|
: payload.failureCategory === "rate-limit"
|
|
? "; primary provider rate limit reached"
|
|
: payload.failureCategory === "model-selection"
|
|
? "; primary model was unavailable"
|
|
: payload.failureCategory === "provider-error"
|
|
? "; primary provider failed"
|
|
: "";
|
|
/*
|
|
FNXC:ModelFallback 2026-07-14-15:58:
|
|
A successful fallback must still explain the primary failure on the task. Persist a bounded category rather than raw provider text so operators can distinguish authentication from capacity/model failures without leaking credentials or arbitrary response bodies into activity logs.
|
|
*/
|
|
return `[fallback] ${label} switched from ${payload.primaryModel} to ${payload.fallbackModel} (${payload.triggerPoint}${reason})`;
|
|
}
|
|
|
|
export function createFallbackModelObserver(options: FallbackModelObserverOptions) {
|
|
return async (payload: FallbackModelUsedPayload): Promise<void> => {
|
|
const taskId = options.taskId ?? payload.taskId;
|
|
const taskTitle = options.taskTitle ?? payload.taskTitle;
|
|
const message = buildFallbackLogMessage(options.label, payload);
|
|
|
|
if (taskId && options.store?.logEntry) {
|
|
await options.store.logEntry(taskId, message).catch(() => undefined);
|
|
}
|
|
if (taskId && options.store?.appendAgentLog) {
|
|
await options.store.appendAgentLog(taskId, message, "status", undefined, options.agent).catch(() => undefined);
|
|
}
|
|
|
|
await notifyFallbackUsed({
|
|
primaryModel: payload.primaryModel,
|
|
fallbackModel: payload.fallbackModel,
|
|
triggerPoint: payload.triggerPoint,
|
|
taskId,
|
|
taskTitle,
|
|
timestamp: payload.timestamp,
|
|
});
|
|
};
|
|
}
|