FN-6876: add editable model pricing refresh
Add editable model pricing overrides and a LiteLLM refresh path for usage cost analytics. - Add persisted model pricing override settings and apply them to token and team analytics cost calculations. - Add a Command Center pricing fetch endpoint that imports chat pricing from LiteLLM and invalidates settings caches. - Add a Global Models pricing editor with manual rows, removal, and one-click fetch support. - Cover override lookup, settings parity, analytics, route behavior, and UI editing with tests and docs. Files changed: .changeset/model-pricing-overrides.md | 5 + docs/architecture.md | 3 +- docs/dashboard-guide.md | 5 +- docs/settings-reference.md | 3 + packages/core/src/__tests__/model-pricing.test.ts | 101 +++++++++++ .../core/src/__tests__/settings-parity.test.ts | 4 + packages/core/src/__tests__/store-settings.test.ts | 25 +++ .../core/src/__tests__/token-analytics.test.ts | 39 +++++ packages/core/src/index.ts | 4 + packages/core/src/model-pricing.ts | 118 +++++++++++-- packages/core/src/settings-schema.ts | 3 + packages/core/src/team-analytics.ts | 17 +- packages/core/src/token-analytics.ts | 19 +- packages/core/src/types.ts | 12 ++ .../dashboard/app/components/SettingsModal.tsx | 2 + .../settings/sections/GlobalModelsSection.tsx | 8 +- .../settings/sections/ModelPricingSection.css | 82 +++++++++ .../settings/sections/ModelPricingSection.test.tsx | 126 ++++++++++++++ .../settings/sections/ModelPricingSection.tsx | 193 +++++++++++++++++++++ .../register-command-center-routes.test.ts | 172 +++++++++++++++++- .../src/routes/register-command-center-routes.ts | 58 +++++++ 21 files changed, 967 insertions(+), 32 deletions(-) Fusion-Task-Id: FN-6876 Fusion-Task-Lineage: 4d4e3b9d-bec4-4e14-b7bc-88f846465566
This commit is contained in:
gsxdsm
5
.changeset/model-pricing-overrides.md
Normal file
5
.changeset/model-pricing-overrides.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"@runfusion/fusion": minor
|
||||
---
|
||||
|
||||
Add editable global model pricing overrides, a one-click LiteLLM pricing refresh, and override-aware Command Center cost estimates.
|
||||
@@ -863,7 +863,8 @@ Key server capabilities:
|
||||
- System stats snapshot and vitest process controls APIs (`GET /api/system-stats`, `POST /api/kill-vitest`) exposing dashboard process/system telemetry (including app CPU percentage and host memory rendered as numeric values, radial gauges, and trend sparklines in the Command Center System area), task/agent aggregates, and manual vitest process termination. Host-memory usage is derived from shared OS-available memory (`process.availableMemory()` with an unreliable `freemem` fallback) rather than raw free pages so macOS inactive/cache memory is not counted as used.
|
||||
- Command Center analytics APIs (`GET /api/command-center/tokens`, `/tools`, `/activity`, `/productivity`, `/team`, `/github`, `/signals`, `/plugin-activations`, `/live`) are project-scoped dashboard routes. `/productivity` reads Lines changed from nullable `task_commit_associations.additions`/`deletions` merge-time or backfilled diff stats, derives estimated `hoursSaved` from that LOC via the exported `HUMAN_LINES_PER_HOUR` rate, and keeps the unavailable sentinel for both fields when no in-range association has stats. `POST /api/command-center/productivity/backfill-loc` is the explicit operator-triggered, dry-run-defaulting local-git backfill for historical NULL stats; it is not run during dashboard rendering or analytics reads. Its `taskDuration` payload aggregates done tasks whose `executionCompletedAt` falls in the selected range, using positive `tasks.cumulativeActiveMs` values for completed count, average, median, p90, and total active execution time; missing qualifying durations remain unavailable rather than zero. `/signals` aggregates real local `incidents` rows for total/open/resolved counts, MTTR, and source/severity/status breakdowns and returns honest empty/unavailable sentinels instead of synthetic signal volume. `/plugin-activations` aggregates persisted plugin/extension load events for the selected range and returns unavailable when no rows exist instead of treating missing history as zero activations.
|
||||
<!-- FNXC:CommandCenter 2026-06-21-00:00: Maintainers need the pricing contract in architecture docs: MODEL_PRICING is hand-maintained, pricingAsOf changes with every rate edit, provider coverage includes OpenAI/Codex/Anthropic/Gemini, and Command Center never guesses or persists costs. -->
|
||||
- Model pricing & cost estimation: Command Center token cost is derived at read time by `packages/core/src/model-pricing.ts`, not fetched from providers at runtime and not persisted as billing truth. Maintainers update the hand-maintained `MODEL_PRICING` table in that file; keys are lowercased `${provider}:${model}` with a bare `:model` fallback for callers that only know the model id. Each entry stores USD per 1M tokens for input, output, cache-read, and cache-write plus a `source` citation. Bump `pricingAsOf` in the same change as any rate edit, because the dashboard surfaces it as the **prices as of** date and marks entries low-confidence after `PRICING_STALE_AFTER_MS` (approximately 180 days / two quarters) relative to that date. Unknown models resolve to `unavailable` rather than a guessed price. The table is curated from provider pricing pages for Anthropic, OpenAI including explicit `openai-codex:*` Codex ids, and Google Gemini; keep provider/model additions in that curated map rather than adding runtime pricing fetches.
|
||||
<!-- FNXC:CommandCenter 2026-06-22-00:00: FN-6876 made pricing operator-editable: user/global overrides and LiteLLM one-click refreshes must take precedence over the built-in table while remaining estimates, not persisted billing truth. -->
|
||||
- Model pricing & cost estimation: Command Center token cost is derived at read time by `packages/core/src/model-pricing.ts` and is not persisted as billing truth. Maintainers still update the built-in `MODEL_PRICING` fallback table in that file; keys are lowercased `${provider}:${model}` with a bare `:model` fallback for callers that only know the model id. Each entry stores USD per 1M tokens for input, output, cache-read, and cache-write plus a `source` citation. Bump `pricingAsOf` in the same change as any built-in rate edit, because the dashboard surfaces it as the **prices as of** date and marks entries low-confidence after `PRICING_STALE_AFTER_MS` (approximately 180 days / two quarters) relative to that date. Global `modelPricingOverrides` from Settings take precedence over built-ins using the same exact-key then bare-model lookup order; `POST /api/command-center/pricing/fetch` is the only dashboard network path and fetches LiteLLM's model pricing JSON on explicit user action, parses it through the pure core parser, persists the resulting overrides with fetched metadata, and leaves the prior overrides intact on fetch/parse failure. Unknown models resolve to `unavailable` rather than a guessed price.
|
||||
- Remote access APIs (`/api/remote/*`) for provider config, activation, tunnel lifecycle, status, token issuance, authenticated URL generation, and QR payload generation
|
||||
- Operational runbook (prereqs/security/troubleshooting): [`docs/remote-access.md`](./remote-access.md)
|
||||
- `/api/remote/tunnel/start`, `/api/remote/tunnel/stop`, and `/api/remote/tunnel/kill-external` cover tunnel lifecycle and external funnel cleanup.
|
||||
|
||||
@@ -213,6 +213,8 @@ Behavior:
|
||||
|
||||
Custom Providers live in **Settings → Authentication → Custom Providers**, inside the **Advanced: Custom Providers** disclosure. Use this section to add user-defined model providers that speak an OpenAI-compatible API, the OpenAI Responses API, an Anthropic-compatible API, or Google Generative AI. After a provider is saved with models, those models become selectable in model dropdowns, including **Settings → Project Models** lanes and workflow model lanes.
|
||||
|
||||
Settings → Global Models also includes **Model pricing overrides** for Command Center estimates. Add or edit rows with lowercased `provider:model` keys (or bare `:model` fallback keys), USD-per-1M token prices for input/output/cache read/cache write, and optional source text. **Fetch LiteLLM pricing** performs an explicit one-click refresh from LiteLLM's published model pricing JSON, replaces the override table only after a successful parse, and records the fetched timestamp/source; failed fetches keep the existing overrides.
|
||||
|
||||
Supported **API type** values match the dropdown in the form:
|
||||
|
||||
- **OpenAI-compatible**
|
||||
@@ -800,7 +802,8 @@ Features:
|
||||
- **Overview controls dashboard** sits at the top of the Overview landing surface on desktop and mobile. It includes AI engine stop/start backed by `globalPause`, live scheduler status from executor stats, range sliders for `maxConcurrent`, `maxTriageConcurrent`, and `maxWorktrees` that persist through `/api/settings`, and a compact theme dropdown with the same color-chip swatches and Shadcn variant list as Settings → Appearance. These controls reuse existing APIs and App-level theme setters; they do not add a new backend route or second theme owner.
|
||||
- **Overview** summarizes token usage/cost, autonomy, active nodes, sessions, agent runs, tasks done, model breadth, and real open signals, and includes the SDLC throughput funnel for the selected range at the bottom of the Overview content in loading, error, empty, and populated states. Its token total and Live activity snapshot token metric refresh on a bounded live cadence and animate number changes while preserving reduced-motion preferences. The sessions card uses the selected-range `ActivityAnalytics.sessions` value already loaded for the overview. The Live activity snapshot also shows the current board-state count for tasks in progress, independent of the selected analytics date range. Overview includes a graph-rich software-factory snapshot with the existing tokens-by-model bar, tool-category bar, real recharts token-share pie, and the daily activity multi-series line chart placed before the daily activity sparkline/trend so the richer line graph sits higher in the chart grid. These reuse the already-loaded tokens, tools, activity, and signals analytics; the signals count comes from `/api/command-center/signals` and renders unavailable (`—`) while the incidents-backed response is loading or unavailable. The chart reveal/glow accents are decorative and disabled when reduced-motion preferences are active. The SDLC completion rate is shown as a radial gauge and is calculated as cohort conversion from in-range triage entrants, so the rate is capped at 100% even when older tasks finish during the range.
|
||||
<!-- FNXC:CommandCenter 2026-06-21-00:00: Command Center cost must read as an estimated, derived value from recorded token counts and the hand-maintained model pricing map; it is never persisted, and the UI must surface prices-as-of, stale low-confidence, and unavailable unknown-model states instead of implying billing truth. -->
|
||||
- **Tokens** breaks down token totals, estimated cost, tasks, and per-model usage. Per-model and per-provider breakdowns use the task's analytics-only actually-used model snapshot when available, so usage from settings-resolved runs appears under the real runtime model instead of `(unknown)` without changing future model resolution; estimated cost uses the same snapshot-first, legacy-fallback model identity so those resolved runs price normally when the model is in the pricing table. Estimated cost is derived at read time from recorded token counts multiplied by a hand-maintained per-model pricing table; it is not persisted, so historical rows stay tied to current maintained prices instead of stale stored billing truth. The Tokens area shows a **prices as of** date for that table, marks pricing older than the staleness threshold as low-confidence, and shows cost unavailable for models with no pricing entry rather than guessing a price. It includes the existing token-usage-over-time chart, an additive recharts multi-series line graph, and a token-share pie backed by the same grouped token analytics; use the granularity control to switch the time-series request between hourly, daily, and weekly buckets. The token total and charts poll on a bounded cadence, keep the previous data visible during refresh, animate decorative count/bar transitions, and disable those animations for reduced-motion users.
|
||||
<!-- FNXC:CommandCenter 2026-06-22-00:00: FN-6876 requires user-maintained/LiteLLM-fetched pricing overrides to feed Tokens and Team estimates immediately without implying provider billing reconciliation. -->
|
||||
- **Tokens** breaks down token totals, estimated cost, tasks, and per-model usage. Per-model and per-provider breakdowns use the task's analytics-only actually-used model snapshot when available, so usage from settings-resolved runs appears under the real runtime model instead of `(unknown)` without changing future model resolution; estimated cost uses the same snapshot-first, legacy-fallback model identity so those resolved runs price normally when the model is in the pricing table. Estimated cost is derived at read time from recorded token counts multiplied by the effective per-model pricing table: Settings → Global Models pricing overrides win first, then the built-in fallback table is used. It is not persisted, so historical rows stay tied to current maintained prices instead of stale stored billing truth. The Tokens area shows a **prices as of** date/source for the effective table, marks pricing older than the staleness threshold as low-confidence, and shows cost unavailable for models with no pricing entry rather than guessing a price. It includes the existing token-usage-over-time chart, an additive recharts multi-series line graph, and a token-share pie backed by the same grouped token analytics; use the granularity control to switch the time-series request between hourly, daily, and weekly buckets. The token total and charts poll on a bounded cadence, keep the previous data visible during refresh, animate decorative count/bar transitions, and disable those animations for reduced-motion users.
|
||||
- **Tools** shows autonomy ratio, tool-call volume, intervention counts, sessions, and tool categories. The area keeps the existing category bar and adds a recharts category-share pie from `ToolAnalytics.byCategory`. There is intentionally no tools line chart yet because `ToolAnalytics` does not expose a per-day tool trend; the dashboard does not fabricate one or call a new endpoint.
|
||||
- **Activity** tracks sessions, messages, active nodes, active agents, agent heartbeat runs, and stickiness. Agent-run sheets show total, active, completed, and failed runs for the selected range, and the Agent runs/day sparkline trends runs by `agentRuns.startedAt`. The area keeps the existing live animated line charts for messages/day, active agents/day, active nodes/day, and combined throughput/day (`messages + active agents + active nodes`), and adds a recharts multi-series line graph for messages, active agents, and agent runs plus an agent-run outcome pie from the existing `agentRuns` split. These charts reuse the existing activity analytics endpoint, refresh on a bounded 15-second cadence while mounted, keep the previous data visible during refreshes, and disable decorative draw-on motion for reduced-motion users.
|
||||
- **Productivity** separates outcome counters (commits and pull requests), task-duration stats, and volume proxies such as modified files, lines changed, and files by language. The task-duration block counts done tasks completed in the selected range and shows average, median, p90, and total active execution time from `cumulativeActiveMs`; when no qualifying duration data exists, duration values render the unavailable `—` sentinel rather than `0`. It keeps the files-by-language bar and adds a language-share pie from `ProductivityAnalytics.byLanguage`. There is intentionally no productivity line chart because the current productivity response has no per-day throughput or completion time series; no new endpoint is called.
|
||||
|
||||
@@ -38,6 +38,9 @@ Defaults from `DEFAULT_GLOBAL_SETTINGS`; key scope from `GLOBAL_SETTINGS_KEYS`.
|
||||
| `dashboardFontScalePct` | `number` | `100` | Dashboard font scale percentage used by Appearance settings. Valid range: `85` to `125`; applied pre-hydration via document root font-size so board typography (column headers/counts, task cards, and quick-entry text) scales with the setting from first paint. |
|
||||
| `defaultProvider` | `string` | `undefined` | Default AI provider. |
|
||||
| `defaultModelId` | `string` | `undefined` | Default AI model ID. |
|
||||
| `modelPricingOverrides` | `Record<string, ModelPricing>` | `undefined` | Optional global Command Center pricing overrides keyed by lowercased `provider:model` or bare `:model`. Values store USD per 1M input, output, cache-read, and cache-write tokens plus optional `source`; they override the built-in pricing table for cost estimates only and are editable in Settings → Global Models. |
|
||||
| `modelPricingFetchedAt` | `string` | `undefined` | ISO timestamp for the last successful one-click pricing refresh from the Settings → Global Models pricing editor. |
|
||||
| `modelPricingSource` | `string` | `undefined` | Source label/URL for the current pricing override set, currently the LiteLLM model pricing JSON when fetched through the dashboard. |
|
||||
| `fallbackProvider` | `string` | `undefined` | Fallback provider when the primary default model hits transient provider failures or model-compatibility/auth-tier rejections. |
|
||||
| `fallbackModelId` | `string` | `undefined` | Fallback model ID (must pair with `fallbackProvider`). |
|
||||
| `defaultThinkingLevel` | `"off" \| "minimal" \| "low" \| "medium" \| "high" \| "xhigh"` | `undefined` | Default reasoning effort for AI sessions. `xhigh` requests maximum reasoning effort; Claude CLI adapters map it to `high` for non-Opus models and `max` for Opus models. If a provider/runtime rejects simultaneous `thinking` and `reasoning_effort` parameters, Fusion retries without the explicit thinking override instead of failing the run. |
|
||||
|
||||
@@ -4,8 +4,10 @@ import {
|
||||
costFor,
|
||||
lookupPricing,
|
||||
MODEL_PRICING,
|
||||
parseLiteLLMPricing,
|
||||
pricingAsOf,
|
||||
PRICING_STALE_AFTER_MS,
|
||||
type ModelPricingOverrides,
|
||||
} from "../model-pricing.js";
|
||||
|
||||
const ZERO = {
|
||||
@@ -150,6 +152,23 @@ describe("model-pricing", () => {
|
||||
});
|
||||
|
||||
describe("lookupPricing", () => {
|
||||
const overrides: ModelPricingOverrides = {
|
||||
"openai:gpt-4o": {
|
||||
inputPer1M: 99,
|
||||
outputPer1M: 199,
|
||||
cacheReadPer1M: 9,
|
||||
cacheWritePer1M: 29,
|
||||
source: "test override",
|
||||
},
|
||||
"acme:unknown-chat": {
|
||||
inputPer1M: 2,
|
||||
outputPer1M: 4,
|
||||
cacheReadPer1M: 1,
|
||||
cacheWritePer1M: 3,
|
||||
source: "test override",
|
||||
},
|
||||
};
|
||||
|
||||
it("resolves by provider:model", () => {
|
||||
expect(
|
||||
lookupPricing({ provider: "openai", model: "gpt-4o" }),
|
||||
@@ -179,6 +198,88 @@ describe("model-pricing", () => {
|
||||
expect(lookupPricing({ model: "" })).toBeUndefined();
|
||||
expect(lookupPricing({ provider: "x", model: "y" })).toBeUndefined();
|
||||
});
|
||||
|
||||
it("prefers overrides over baseline and keeps baseline fallback", () => {
|
||||
expect(lookupPricing({ provider: "openai", model: "gpt-4o" }, overrides)).toBe(overrides["openai:gpt-4o"]);
|
||||
expect(lookupPricing({ provider: "anthropic", model: "claude-opus-4-8" }, overrides)).toBe(
|
||||
MODEL_PRICING["anthropic:claude-opus-4-8"],
|
||||
);
|
||||
});
|
||||
|
||||
it("resolves overrides for otherwise unknown models", () => {
|
||||
expect(lookupPricing({ provider: "acme", model: "unknown-chat" }, overrides)).toBe(overrides["acme:unknown-chat"]);
|
||||
const result = costFor(
|
||||
{ ...ZERO, inputTokens: 1_000_000, outputTokens: 500_000 },
|
||||
{ provider: "acme", model: "unknown-chat" },
|
||||
undefined,
|
||||
overrides,
|
||||
);
|
||||
expect(result).toMatchObject({ unavailable: false, stale: false });
|
||||
expect(result.usd).toBeCloseTo(4, 2);
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseLiteLLMPricing", () => {
|
||||
it("maps chat rows and cache costs from the LiteLLM schema", () => {
|
||||
const parsed = parseLiteLLMPricing({
|
||||
sample_spec: { mode: "chat" },
|
||||
"gpt-test": {
|
||||
litellm_provider: "openai",
|
||||
mode: "chat",
|
||||
input_cost_per_token: 0.000001,
|
||||
output_cost_per_token: 0.000002,
|
||||
cache_read_input_token_cost: 0.00000025,
|
||||
cache_creation_input_token_cost: 0.00000125,
|
||||
},
|
||||
"claude-test": {
|
||||
litellm_provider: "anthropic",
|
||||
mode: "chat",
|
||||
input_cost_per_token: 0.000003,
|
||||
output_cost_per_token: 0.000015,
|
||||
},
|
||||
"gemini-test": {
|
||||
litellm_provider: "vertex_ai-language-models",
|
||||
mode: "chat",
|
||||
input_cost_per_token: 0.0000005,
|
||||
output_cost_per_token: 0.0000015,
|
||||
},
|
||||
"embedding-test": {
|
||||
litellm_provider: "openai",
|
||||
mode: "embedding",
|
||||
input_cost_per_token: 0.000001,
|
||||
output_cost_per_token: 0.000002,
|
||||
},
|
||||
"missing-output": {
|
||||
litellm_provider: "openai",
|
||||
mode: "chat",
|
||||
input_cost_per_token: 0.000001,
|
||||
},
|
||||
});
|
||||
|
||||
expect(parsed.count).toBe(3);
|
||||
expect(parsed.overrides["openai:gpt-test"]).toEqual({
|
||||
inputPer1M: 1,
|
||||
outputPer1M: 2,
|
||||
cacheReadPer1M: 0.25,
|
||||
cacheWritePer1M: 1.25,
|
||||
source: "litellm/model_prices_and_context_window.json",
|
||||
});
|
||||
expect(parsed.overrides["anthropic:claude-test"]).toMatchObject({
|
||||
inputPer1M: 3,
|
||||
outputPer1M: 15,
|
||||
cacheReadPer1M: 3,
|
||||
cacheWritePer1M: 3,
|
||||
});
|
||||
expect(parsed.overrides["google:gemini-test"]).toMatchObject({ inputPer1M: 0.5, outputPer1M: 1.5 });
|
||||
expect(parsed.overrides).not.toHaveProperty("openai:embedding-test");
|
||||
expect(parsed.overrides).not.toHaveProperty("openai:missing-output");
|
||||
});
|
||||
|
||||
it("returns an empty map for malformed input", () => {
|
||||
expect(parseLiteLLMPricing(null)).toEqual({ overrides: {}, count: 0 });
|
||||
expect(parseLiteLLMPricing([])).toEqual({ overrides: {}, count: 0 });
|
||||
expect(parseLiteLLMPricing({ "gpt-test": "bad" })).toEqual({ overrides: {}, count: 0 });
|
||||
});
|
||||
});
|
||||
|
||||
it("seeds Anthropic, OpenAI Codex, OpenAI, and Google providers", () => {
|
||||
|
||||
@@ -73,6 +73,10 @@ describe("settings key parity", () => {
|
||||
expect(isProjectSettingsKey("persistAgentThinkingLogEphemeral")).toBe(false);
|
||||
expect(isGlobalOnlySettingsKey("persistAgentThinkingLogEphemeral")).toBe(true);
|
||||
expect(isGlobalSettingsKey("researchSettings")).toBe(false);
|
||||
expect(isGlobalSettingsKey("modelPricingOverrides")).toBe(true);
|
||||
expect(isGlobalSettingsKey("modelPricingFetchedAt")).toBe(true);
|
||||
expect(isGlobalSettingsKey("modelPricingSource")).toBe(true);
|
||||
expect(isProjectSettingsKey("modelPricingOverrides")).toBe(false);
|
||||
expect(isGlobalSettingsKey("agentMemoryInclusionMode")).toBe(true);
|
||||
expect(isProjectSettingsKey("agentMemoryInclusionMode")).toBe(false);
|
||||
});
|
||||
|
||||
@@ -1425,6 +1425,31 @@ describe("TaskStore", () => {
|
||||
expect(settings.defaultModelId).toBe("gpt-4o");
|
||||
});
|
||||
|
||||
it("round-trips model pricing settings through global scope", async () => {
|
||||
await harness.store().updateGlobalSettings({
|
||||
modelPricingOverrides: {
|
||||
"openai:gpt-4o": {
|
||||
inputPer1M: 1,
|
||||
outputPer1M: 2,
|
||||
cacheReadPer1M: 0.5,
|
||||
cacheWritePer1M: 1,
|
||||
source: "test",
|
||||
},
|
||||
},
|
||||
modelPricingFetchedAt: "2026-06-22T00:00:00.000Z",
|
||||
modelPricingSource: "litellm/model_prices_and_context_window.json",
|
||||
});
|
||||
|
||||
const settings = await harness.store().getSettings();
|
||||
expect(settings.modelPricingOverrides?.["openai:gpt-4o"]?.outputPer1M).toBe(2);
|
||||
expect(settings.modelPricingFetchedAt).toBe("2026-06-22T00:00:00.000Z");
|
||||
expect(settings.modelPricingSource).toBe("litellm/model_prices_and_context_window.json");
|
||||
|
||||
const { global, project } = await harness.store().getSettingsByScope();
|
||||
expect(global.modelPricingOverrides).toEqual(settings.modelPricingOverrides);
|
||||
expect(project.modelPricingOverrides).toBeUndefined();
|
||||
});
|
||||
|
||||
it("updateGlobalSettings emits settings:updated event", async () => {
|
||||
const events: Array<{ settings: any; previous: any }> = [];
|
||||
harness.store().on("settings:updated", (data) => events.push(data));
|
||||
|
||||
@@ -473,6 +473,45 @@ describe("token-analytics", () => {
|
||||
expect(groups.get("mystery-model")?.cost).toEqual({ usd: null, unavailable: true, stale: false });
|
||||
});
|
||||
|
||||
it("applies pricing overrides while preserving baseline fallback", () => {
|
||||
insertTask(db, {
|
||||
id: "override-priced",
|
||||
inputTokens: 1_000_000,
|
||||
outputTokens: 1_000_000,
|
||||
totalTokens: 2_000_000,
|
||||
lastUsedAt: "2026-03-01T00:00:00.000Z",
|
||||
modelProvider: "openai",
|
||||
modelId: "gpt-4o",
|
||||
});
|
||||
insertTask(db, {
|
||||
id: "baseline-priced",
|
||||
inputTokens: 1_000_000,
|
||||
outputTokens: 1_000_000,
|
||||
totalTokens: 2_000_000,
|
||||
lastUsedAt: "2026-03-02T00:00:00.000Z",
|
||||
modelProvider: "anthropic",
|
||||
modelId: "claude-opus-4-8",
|
||||
});
|
||||
|
||||
const result = aggregateTokenAnalytics(db, {
|
||||
groupBy: "model",
|
||||
pricingOverrides: {
|
||||
"openai:gpt-4o": {
|
||||
inputPer1M: 1,
|
||||
outputPer1M: 2,
|
||||
cacheReadPer1M: 1,
|
||||
cacheWritePer1M: 1,
|
||||
source: "test override",
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const groups = new Map(result.groups.map((group) => [group.key, group.cost]));
|
||||
expect(groups.get("gpt-4o")?.usd).toBeCloseTo(3, 2);
|
||||
expect(groups.get("claude-opus-4-8")?.usd).toBeCloseTo(30, 2);
|
||||
expect(result.cost.usd).toBeCloseTo(33, 2);
|
||||
});
|
||||
|
||||
it("returns an empty series for an empty requested range", () => {
|
||||
insertTask(db, { id: "t1", inputTokens: 100, totalTokens: 100, lastUsedAt: "2026-03-01T00:00:00.000Z", modelId: "model-A" });
|
||||
|
||||
|
||||
@@ -546,12 +546,16 @@ export type {
|
||||
export {
|
||||
costFor,
|
||||
lookupPricing,
|
||||
parseLiteLLMPricing,
|
||||
MODEL_PRICING,
|
||||
LITELLM_PRICING_SOURCE_LABEL,
|
||||
LITELLM_PRICING_SOURCE_URL,
|
||||
pricingAsOf,
|
||||
PRICING_STALE_AFTER_MS,
|
||||
} from "./model-pricing.js";
|
||||
export type {
|
||||
ModelPricing,
|
||||
ModelPricingOverrides,
|
||||
ModelRef,
|
||||
UsageForCost,
|
||||
CostResult,
|
||||
|
||||
@@ -2,17 +2,20 @@
|
||||
* Model pricing → USD cost derivation (KTD6, U3).
|
||||
*
|
||||
* Cost is **derived at read time** from token counts × a hand-maintained
|
||||
* pricing map; it is never persisted (so historical rows stay correct when
|
||||
* prices change, and no backfill migration is needed). Unknown models surface
|
||||
* tokens with cost marked `unavailable` rather than guessing a price.
|
||||
* pricing map plus optional user-managed overrides; it is never persisted (so
|
||||
* historical rows stay correct when prices change, and no backfill migration is
|
||||
* needed). Unknown models surface tokens with cost marked `unavailable` rather
|
||||
* than guessing a price.
|
||||
*
|
||||
* ⚠️ HAND-MAINTAINED MAP. The `MODEL_PRICING` table below is curated by humans
|
||||
* from each provider's public pricing pages — it is NOT fetched at runtime.
|
||||
* When you update a rate, bump {@link pricingAsOf} in the same change. The UI
|
||||
* surfaces `pricingAsOf` ("prices as of <date>") and marks entries older than
|
||||
* {@link PRICING_STALE_AFTER_MS} as low-confidence, so stale-but-present rates
|
||||
* (which the unknown-model guard does not catch) are visible rather than
|
||||
* silently wrong.
|
||||
* Callers may supply persisted overrides, including entries parsed from the
|
||||
* canonical LiteLLM dataset, and those overrides take precedence over this
|
||||
* baseline. When you update a baseline rate, bump {@link pricingAsOf} in the
|
||||
* same change. The UI surfaces `pricingAsOf` ("prices as of <date>") and marks
|
||||
* entries older than {@link PRICING_STALE_AFTER_MS} as low-confidence, so
|
||||
* stale-but-present rates (which the unknown-model guard does not catch) are
|
||||
* visible rather than silently wrong.
|
||||
*
|
||||
* Rates are USD **per 1,000,000 tokens**.
|
||||
*
|
||||
@@ -35,6 +38,15 @@ export const pricingAsOf = "2026-06-21";
|
||||
*/
|
||||
export const PRICING_STALE_AFTER_MS = 180 * 24 * 60 * 60 * 1000;
|
||||
|
||||
/*
|
||||
* FNXC:CommandCenter 2026-06-22-00:00:
|
||||
* Users need one-click pricing refreshes from LiteLLM's continuously updated community dataset while the core module remains pure. Keep the URL as data only; dashboard routes own HTTP, validation errors, and persistence.
|
||||
*/
|
||||
export const LITELLM_PRICING_SOURCE_URL =
|
||||
"https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json";
|
||||
|
||||
export const LITELLM_PRICING_SOURCE_LABEL = "litellm/model_prices_and_context_window.json";
|
||||
|
||||
/** A single model's per-1M-token rates plus a citation. */
|
||||
export interface ModelPricing {
|
||||
/** USD per 1M uncached input tokens. */
|
||||
@@ -49,6 +61,9 @@ export interface ModelPricing {
|
||||
source: string;
|
||||
}
|
||||
|
||||
/** User-managed pricing overrides keyed by lowercased `provider:model`. */
|
||||
export type ModelPricingOverrides = Record<string, ModelPricing>;
|
||||
|
||||
/** Token counts to price. Mirrors {@link TokenTotals} from token-analytics. */
|
||||
export interface UsageForCost {
|
||||
inputTokens: number;
|
||||
@@ -317,24 +332,92 @@ function normalize(s: string | null | undefined): string {
|
||||
return (s ?? "").trim().toLowerCase();
|
||||
}
|
||||
|
||||
function findBareModelPricing(
|
||||
model: string,
|
||||
entries: Record<string, ModelPricing> | Readonly<Record<string, ModelPricing>>,
|
||||
): ModelPricing | undefined {
|
||||
for (const [key, entry] of Object.entries(entries)) {
|
||||
if (key.endsWith(`:${model}`)) return entry;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a pricing entry for a model. Tries `provider:model` first, then the
|
||||
* bare `:model` (provider-agnostic) fallback. Returns `undefined` for unknown
|
||||
* models — callers must treat that as `unavailable`, never as a guessed price.
|
||||
* Resolve a pricing entry for a model. Tries override `provider:model` first,
|
||||
* then override bare-model fallback, then the built-in baseline using the same
|
||||
* precedence. Returns `undefined` for unknown models — callers must treat that
|
||||
* as `unavailable`, never as a guessed price.
|
||||
*
|
||||
* FNXC:CommandCenter 2026-06-22-00:00:
|
||||
* Editable/fetched model rates must override the hand-maintained baseline without removing the baseline fallback. Keep exact provider:model checks before bare-model scans so provider-specific overrides stay deterministic.
|
||||
*/
|
||||
export function lookupPricing(ref: ModelRef): ModelPricing | undefined {
|
||||
export function lookupPricing(ref: ModelRef, overrides?: ModelPricingOverrides): ModelPricing | undefined {
|
||||
const provider = normalize(ref.provider);
|
||||
const model = normalize(ref.model);
|
||||
if (!model) return undefined;
|
||||
if (provider) {
|
||||
const exactOverride = overrides?.[`${provider}:${model}`];
|
||||
if (exactOverride) return exactOverride;
|
||||
}
|
||||
const bareOverride = overrides ? findBareModelPricing(model, overrides) : undefined;
|
||||
if (bareOverride) return bareOverride;
|
||||
if (provider) {
|
||||
const exact = MODEL_PRICING[`${provider}:${model}`];
|
||||
if (exact) return exact;
|
||||
}
|
||||
// Provider-agnostic fallback: scan for any entry whose model id matches.
|
||||
for (const [key, entry] of Object.entries(MODEL_PRICING)) {
|
||||
if (key.endsWith(`:${model}`)) return entry;
|
||||
return findBareModelPricing(model, MODEL_PRICING);
|
||||
}
|
||||
|
||||
function litellmProviderToFusionProvider(provider: unknown): string | null {
|
||||
if (typeof provider !== "string") return null;
|
||||
const normalized = normalize(provider);
|
||||
if (normalized === "openai") return "openai";
|
||||
if (normalized === "anthropic") return "anthropic";
|
||||
if (normalized === "gemini" || normalized.startsWith("gemini")) return "google";
|
||||
if (normalized === "vertex_ai" || normalized.startsWith("vertex_ai")) return "google";
|
||||
if (normalized === "vertex_ai-language-models") return "google";
|
||||
return null;
|
||||
}
|
||||
|
||||
function numericField(entry: Record<string, unknown>, key: string): number | undefined {
|
||||
const value = entry[key];
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse LiteLLM's canonical pricing dataset into Fusion pricing overrides.
|
||||
* Pure: no HTTP, DB access, or clock reads. Unsupported providers and non-chat
|
||||
* rows are skipped so a broad upstream dataset can safely feed Fusion's known
|
||||
* model-provider surface.
|
||||
*/
|
||||
export function parseLiteLLMPricing(json: unknown): { overrides: ModelPricingOverrides; count: number } {
|
||||
const overrides: ModelPricingOverrides = {};
|
||||
if (json === null || typeof json !== "object" || Array.isArray(json)) {
|
||||
return { overrides, count: 0 };
|
||||
}
|
||||
return undefined;
|
||||
for (const [modelId, value] of Object.entries(json as Record<string, unknown>)) {
|
||||
if (modelId === "sample_spec") continue;
|
||||
if (value === null || typeof value !== "object" || Array.isArray(value)) continue;
|
||||
const entry = value as Record<string, unknown>;
|
||||
if (entry.mode !== "chat") continue;
|
||||
const provider = litellmProviderToFusionProvider(entry.litellm_provider);
|
||||
if (!provider) continue;
|
||||
const inputCost = numericField(entry, "input_cost_per_token");
|
||||
const outputCost = numericField(entry, "output_cost_per_token");
|
||||
if (inputCost === undefined || outputCost === undefined) continue;
|
||||
const inputPer1M = inputCost * 1_000_000;
|
||||
const outputPer1M = outputCost * 1_000_000;
|
||||
const cacheRead = numericField(entry, "cache_read_input_token_cost");
|
||||
const cacheWrite = numericField(entry, "cache_creation_input_token_cost");
|
||||
overrides[`${provider}:${normalize(modelId)}`] = {
|
||||
inputPer1M,
|
||||
outputPer1M,
|
||||
cacheReadPer1M: cacheRead === undefined ? inputPer1M : cacheRead * 1_000_000,
|
||||
cacheWritePer1M: cacheWrite === undefined ? inputPer1M : cacheWrite * 1_000_000,
|
||||
source: LITELLM_PRICING_SOURCE_LABEL,
|
||||
};
|
||||
}
|
||||
return { overrides, count: Object.keys(overrides).length };
|
||||
}
|
||||
|
||||
/** True when the pricing map is older than the threshold relative to `now`. */
|
||||
@@ -359,9 +442,10 @@ export function costFor(
|
||||
usage: UsageForCost,
|
||||
model: ModelRef,
|
||||
now?: number,
|
||||
overrides?: ModelPricingOverrides,
|
||||
): CostResult {
|
||||
const stale = isStale(now);
|
||||
const pricing = lookupPricing(model);
|
||||
const pricing = lookupPricing(model, overrides);
|
||||
if (!pricing) {
|
||||
return { usd: null, unavailable: true, stale };
|
||||
}
|
||||
|
||||
@@ -75,6 +75,9 @@ export const DEFAULT_GLOBAL_SETTINGS = {
|
||||
defaultProvider: undefined,
|
||||
defaultModelId: undefined,
|
||||
testMode: undefined,
|
||||
modelPricingOverrides: undefined,
|
||||
modelPricingFetchedAt: undefined,
|
||||
modelPricingSource: undefined,
|
||||
modelRouterEnabled: undefined,
|
||||
modelRouterCheapProvider: undefined,
|
||||
modelRouterCheapModelId: undefined,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { Database } from "./db.js";
|
||||
import { costFor, type CostResult } from "./model-pricing.js";
|
||||
import { costFor, type CostResult, type ModelPricingOverrides } from "./model-pricing.js";
|
||||
import type { TokenTotals } from "./token-analytics.js";
|
||||
|
||||
export interface TeamAnalyticsQuery {
|
||||
@@ -9,6 +9,8 @@ export interface TeamAnalyticsQuery {
|
||||
to?: string;
|
||||
/** Epoch ms "now" used only for pricing-staleness. */
|
||||
now?: number;
|
||||
/** User-managed pricing overrides that take precedence over the built-in baseline. */
|
||||
pricingOverrides?: ModelPricingOverrides;
|
||||
}
|
||||
|
||||
export interface TeamMetricTotals {
|
||||
@@ -106,7 +108,12 @@ function addTokenRow(totals: TokenTotals, row: TaskTokenRow): void {
|
||||
totals.nTasks += 1;
|
||||
}
|
||||
|
||||
function addRowCost(acc: CostAccumulator, row: TaskTokenRow, now?: number): void {
|
||||
function addRowCost(
|
||||
acc: CostAccumulator,
|
||||
row: TaskTokenRow,
|
||||
now?: number,
|
||||
pricingOverrides?: ModelPricingOverrides,
|
||||
): void {
|
||||
const result = costFor(
|
||||
{
|
||||
inputTokens: row.inputTokens ?? 0,
|
||||
@@ -116,6 +123,7 @@ function addRowCost(acc: CostAccumulator, row: TaskTokenRow, now?: number): void
|
||||
},
|
||||
{ provider: row.modelProvider, model: row.modelId },
|
||||
now,
|
||||
pricingOverrides,
|
||||
);
|
||||
if (result.stale) acc.anyStale = true;
|
||||
if (result.unavailable || result.usd === null) {
|
||||
@@ -188,6 +196,7 @@ export function aggregateTeamAnalytics(
|
||||
const costAccumulators = new Map<string, CostAccumulator>();
|
||||
const totalTokens = emptyTokenTotals();
|
||||
const totalCost = emptyCostAccumulator();
|
||||
const pricingOverrides = query.pricingOverrides;
|
||||
|
||||
const agents = db
|
||||
.prepare(`SELECT id, name, role, state FROM agents ORDER BY id`)
|
||||
@@ -231,8 +240,8 @@ export function aggregateTeamAnalytics(
|
||||
costAccumulators.set(row.agentId, agentCost);
|
||||
addTokenRow(summary.tokens, row);
|
||||
addTokenRow(totalTokens, row);
|
||||
addRowCost(agentCost, row, query.now);
|
||||
addRowCost(totalCost, row, query.now);
|
||||
addRowCost(agentCost, row, query.now, pricingOverrides);
|
||||
addRowCost(totalCost, row, query.now, pricingOverrides);
|
||||
}
|
||||
|
||||
const completedClauses = ["assignedAgentId IS NOT NULL", `"column" = 'done'`, "columnMovedAt IS NOT NULL"];
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { Database } from "./db.js";
|
||||
import { costFor, type CostResult } from "./model-pricing.js";
|
||||
import { costFor, type CostResult, type ModelPricingOverrides } from "./model-pricing.js";
|
||||
import type { TaskTokenUsagePerModel } from "./types.js";
|
||||
|
||||
/**
|
||||
@@ -85,6 +85,8 @@ export interface TokenAnalyticsQuery {
|
||||
* cost is never marked stale. Pure: the module never reads the clock itself.
|
||||
*/
|
||||
now?: number;
|
||||
/** User-managed pricing overrides that take precedence over the built-in baseline. */
|
||||
pricingOverrides?: ModelPricingOverrides;
|
||||
}
|
||||
|
||||
function emptyTotals(): TokenTotals {
|
||||
@@ -149,7 +151,12 @@ function emptyCostAccumulator(): CostAccumulator {
|
||||
return { usd: 0, anyPriced: false, anyUnavailable: false, anyStale: false };
|
||||
}
|
||||
|
||||
function addRowCost(acc: CostAccumulator, row: TaskTokenRow, now?: number): void {
|
||||
function addRowCost(
|
||||
acc: CostAccumulator,
|
||||
row: TaskTokenRow,
|
||||
now?: number,
|
||||
pricingOverrides?: ModelPricingOverrides,
|
||||
): void {
|
||||
/*
|
||||
* FNXC:CommandCenter 2026-06-18-12:00:
|
||||
* Token cost attribution must use the actually-used model snapshot first, then legacy own-model columns, matching groupKeyFor so resolved-via-settings tasks show priced Command Center costs instead of unavailable groups.
|
||||
@@ -166,6 +173,7 @@ function addRowCost(acc: CostAccumulator, row: TaskTokenRow, now?: number): void
|
||||
model: row.tokenUsageModelId ?? row.modelId,
|
||||
},
|
||||
now,
|
||||
pricingOverrides,
|
||||
);
|
||||
if (result.stale) acc.anyStale = true;
|
||||
if (result.unavailable || result.usd === null) {
|
||||
@@ -308,10 +316,11 @@ export function aggregateTokenAnalytics(
|
||||
const groupBy = query.groupBy;
|
||||
const granularity = query.granularity;
|
||||
const now = query.now;
|
||||
const pricingOverrides = query.pricingOverrides;
|
||||
|
||||
for (const row of rows) {
|
||||
addRow(totals, row);
|
||||
addRowCost(totalCost, row, now);
|
||||
addRowCost(totalCost, row, now, pricingOverrides);
|
||||
if (groupBy) {
|
||||
const groupRows = (groupBy === "model" || groupBy === "provider") ? parsePerModelRows(row) : [];
|
||||
const rowsForGroup = groupRows.length > 0 ? groupRows : [row];
|
||||
@@ -324,7 +333,7 @@ export function aggregateTokenAnalytics(
|
||||
groupCostMap.set(key, emptyCostAccumulator());
|
||||
}
|
||||
addRow(group, groupRow);
|
||||
addRowCost(groupCostMap.get(key)!, groupRow, now);
|
||||
addRowCost(groupCostMap.get(key)!, groupRow, now, pricingOverrides);
|
||||
}
|
||||
}
|
||||
if (granularity) {
|
||||
@@ -336,7 +345,7 @@ export function aggregateTokenAnalytics(
|
||||
seriesCostMap.set(bucket, emptyCostAccumulator());
|
||||
}
|
||||
addRow(point, row);
|
||||
addRowCost(seriesCostMap.get(bucket)!, row, now);
|
||||
addRowCost(seriesCostMap.get(bucket)!, row, now, pricingOverrides);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { InReviewStallSignal } from "./in-review-stall.js";
|
||||
import type { ModelPricing } from "./model-pricing.js";
|
||||
import type { InReviewStalledSignal } from "./in-review-stalled.js";
|
||||
import type { StalePausedReviewSignal } from "./stale-paused-review.js";
|
||||
import type { StalePausedTodoSignal } from "./stale-paused-todo.js";
|
||||
@@ -2978,6 +2979,17 @@ export interface GlobalSettings {
|
||||
* of per-task or per-lane overrides. No network calls, zero token cost.
|
||||
* Project `testMode` takes precedence over the global value. */
|
||||
testMode?: boolean;
|
||||
/**
|
||||
* User-edited or one-click-fetched pricing entries keyed by lowercased `provider:model`.
|
||||
*
|
||||
* FNXC:CommandCenter 2026-06-22-00:00:
|
||||
* Global pricing overrides let Command Center cost estimates reflect user-maintained or LiteLLM-refreshed rates while preserving the built-in MODEL_PRICING fallback for unedited models.
|
||||
*/
|
||||
modelPricingOverrides?: Record<string, ModelPricing>;
|
||||
/** ISO timestamp for the last successful pricing refresh from the configured source. */
|
||||
modelPricingFetchedAt?: string;
|
||||
/** Source label or URL for the current global pricing override set. */
|
||||
modelPricingSource?: string;
|
||||
/** Fusion Model Router opt-in (U17/KTD9). When true, a conservative selection
|
||||
* layer may down-route an allowlist of mechanical steps (dependabot bumps,
|
||||
* lint-only fixes) to a cheap model tier before a session starts; everything
|
||||
|
||||
@@ -2631,6 +2631,8 @@ export function SettingsModal({
|
||||
favoriteModels={favoriteModels}
|
||||
onToggleFavorite={handleToggleFavorite}
|
||||
onToggleModelFavorite={handleToggleModelFavorite}
|
||||
addToast={addToast}
|
||||
projectId={projectId}
|
||||
/>
|
||||
);
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@ import { useTranslation } from "react-i18next";
|
||||
import { THINKING_LEVELS } from "@fusion/core";
|
||||
import type { Settings, ThinkingLevel } from "@fusion/core";
|
||||
import type { ModelInfo } from "../../../api";
|
||||
import type { ToastType } from "../../../hooks/useToast";
|
||||
import { ModelPricingSection } from "./ModelPricingSection";
|
||||
import { CustomModelDropdown } from "../../CustomModelDropdown";
|
||||
import type { SectionBaseProps, ModelLane } from "./context";
|
||||
import { LoadingSpinner } from "../../LoadingSpinner";
|
||||
@@ -22,8 +24,10 @@ export interface GlobalModelsSectionProps extends SectionBaseProps {
|
||||
favoriteModels: string[];
|
||||
onToggleFavorite: (provider: string) => void;
|
||||
onToggleModelFavorite: (modelId: string) => void;
|
||||
addToast: (message: string, type?: ToastType) => void;
|
||||
projectId?: string;
|
||||
}
|
||||
export function GlobalModelsSection({ scopeBanner, form, setForm, availableModels, modelsLoading, globalModelLanes, favoriteProviders, favoriteModels, onToggleFavorite, onToggleModelFavorite, }: GlobalModelsSectionProps) {
|
||||
export function GlobalModelsSection({ scopeBanner, form, setForm, availableModels, modelsLoading, globalModelLanes, favoriteProviders, favoriteModels, onToggleFavorite, onToggleModelFavorite, addToast, projectId, }: GlobalModelsSectionProps) {
|
||||
const { t } = useTranslation("app");
|
||||
const selectedValue = form.defaultProvider && form.defaultModelId
|
||||
? `${form.defaultProvider}/${form.defaultModelId}`
|
||||
@@ -122,6 +126,8 @@ export function GlobalModelsSection({ scopeBanner, form, setForm, availableModel
|
||||
})}
|
||||
</>)}
|
||||
|
||||
<ModelPricingSection form={form} setForm={setForm} addToast={addToast} projectId={projectId}/>
|
||||
|
||||
{/* --- Startup Model Sync --- */}
|
||||
<h4 className="settings-section-heading settings-section-heading--spaced">{t("settings.globalModels.startupModelSync", "Startup Model Sync")}</h4>
|
||||
<div className="form-group">
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
.model-pricing-section {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: var(--spacing-md, var(--space-md));
|
||||
}
|
||||
|
||||
.model-pricing-section__header {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
justify-content: space-between;
|
||||
gap: var(--spacing-md, var(--space-md));
|
||||
}
|
||||
|
||||
.model-pricing-section__meta {
|
||||
margin: var(--spacing-xs, var(--space-xs)) 0 0;
|
||||
}
|
||||
|
||||
.model-pricing-table {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: var(--spacing-xs, var(--space-xs));
|
||||
overflow-x: auto;
|
||||
}
|
||||
|
||||
.model-pricing-row {
|
||||
display: grid;
|
||||
grid-template-columns:
|
||||
minmax(12rem, 1.4fr)
|
||||
minmax(7rem, 1fr)
|
||||
minmax(7rem, 1fr)
|
||||
minmax(7rem, 1fr)
|
||||
minmax(7rem, 1fr)
|
||||
minmax(10rem, 1.2fr)
|
||||
minmax(5rem, auto);
|
||||
gap: var(--spacing-xs, var(--space-xs));
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.model-pricing-row--head {
|
||||
color: var(--text-muted);
|
||||
font-size: var(--font-size-xs, 0.75rem);
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.model-pricing-row--add {
|
||||
padding-top: var(--spacing-xs, var(--space-xs));
|
||||
border-top: thin solid var(--border);
|
||||
}
|
||||
|
||||
.model-pricing-key {
|
||||
overflow-wrap: anywhere;
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
.model-pricing-empty {
|
||||
grid-column: 1 / -1;
|
||||
}
|
||||
|
||||
@media (max-width: 768px) {
|
||||
.model-pricing-section__header {
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.model-pricing-section__header .btn {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.model-pricing-row {
|
||||
grid-template-columns: minmax(14rem, 1fr);
|
||||
padding: var(--spacing-sm, var(--space-sm));
|
||||
border: thin solid var(--border);
|
||||
border-radius: var(--radius-md);
|
||||
}
|
||||
|
||||
.model-pricing-row--head {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.model-pricing-row--add {
|
||||
border-top: thin solid var(--border);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
|
||||
import { useState } from "react";
|
||||
import { ModelPricingSection } from "./ModelPricingSection";
|
||||
import type { SettingsFormState } from "./context";
|
||||
|
||||
const { apiMock } = vi.hoisted(() => ({
|
||||
apiMock: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("react-i18next", () => ({
|
||||
useTranslation: () => ({ t: (_key: string, fallback: string, vars?: Record<string, unknown>) => {
|
||||
if (!vars) return fallback;
|
||||
return Object.entries(vars).reduce((text, [key, value]) => text.replace(`{{${key}}}`, String(value)), fallback);
|
||||
} }),
|
||||
}));
|
||||
|
||||
// Mock the API helper so fetch actions stay deterministic and do not hit the dashboard server.
|
||||
vi.mock("../../../api", () => ({
|
||||
api: apiMock,
|
||||
}));
|
||||
|
||||
function Harness({ initial, addToast = vi.fn() }: { initial: SettingsFormState; addToast?: (message: string, type?: "success" | "error" | "info" | "warning") => void }) {
|
||||
const [form, setForm] = useState<SettingsFormState>(initial);
|
||||
return (
|
||||
<ModelPricingSection
|
||||
form={form}
|
||||
setForm={setForm}
|
||||
addToast={addToast}
|
||||
projectId="proj-a"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
const initialForm = (): SettingsFormState => ({
|
||||
modelPricingOverrides: {
|
||||
"openai:gpt-4o": {
|
||||
inputPer1M: 2.5,
|
||||
outputPer1M: 10,
|
||||
cacheReadPer1M: 1.25,
|
||||
cacheWritePer1M: 2.5,
|
||||
source: "manual",
|
||||
},
|
||||
},
|
||||
} as SettingsFormState);
|
||||
|
||||
describe("ModelPricingSection", () => {
|
||||
beforeEach(() => {
|
||||
apiMock.mockReset();
|
||||
});
|
||||
|
||||
it("renders existing overrides and edits a row", () => {
|
||||
render(<Harness initial={initialForm()} />);
|
||||
|
||||
expect(screen.getByText("openai:gpt-4o")).toBeInTheDocument();
|
||||
const inputRate = screen.getByLabelText("openai:gpt-4o input per 1M");
|
||||
fireEvent.change(inputRate, { target: { value: "3.75" } });
|
||||
|
||||
expect(screen.getByLabelText("openai:gpt-4o input per 1M")).toHaveValue(3.75);
|
||||
});
|
||||
|
||||
it("adds and deletes pricing rows through form state", () => {
|
||||
render(<Harness initial={{} as SettingsFormState} />);
|
||||
|
||||
fireEvent.change(screen.getByLabelText("New provider:model key"), { target: { value: "Anthropic:Claude-Test" } });
|
||||
fireEvent.change(screen.getByLabelText("New input rate"), { target: { value: "1" } });
|
||||
fireEvent.change(screen.getByLabelText("New output rate"), { target: { value: "5" } });
|
||||
fireEvent.change(screen.getByLabelText("New cache read rate"), { target: { value: "0.1" } });
|
||||
fireEvent.change(screen.getByLabelText("New cache write rate"), { target: { value: "1.25" } });
|
||||
fireEvent.change(screen.getByLabelText("New source"), { target: { value: "manual-test" } });
|
||||
fireEvent.click(screen.getByRole("button", { name: "Add row" }));
|
||||
|
||||
expect(screen.getByText("anthropic:claude-test")).toBeInTheDocument();
|
||||
expect(screen.getByLabelText("anthropic:claude-test output per 1M")).toHaveValue(5);
|
||||
|
||||
fireEvent.click(screen.getByRole("button", { name: "Delete" }));
|
||||
expect(screen.queryByText("anthropic:claude-test")).not.toBeInTheDocument();
|
||||
expect(screen.getByText("No model pricing overrides yet. Add one manually or fetch the latest LiteLLM prices.")).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("shows an error toast and resets loading when pricing fetch fails", async () => {
|
||||
const addToast = vi.fn();
|
||||
let rejectFetch: (error: Error) => void = () => undefined;
|
||||
apiMock.mockImplementationOnce(() => new Promise((_resolve, reject) => {
|
||||
rejectFetch = reject;
|
||||
}));
|
||||
|
||||
render(<Harness initial={{} as SettingsFormState} addToast={addToast} />);
|
||||
fireEvent.click(screen.getByRole("button", { name: "Fetch latest prices" }));
|
||||
|
||||
expect(await screen.findByRole("button", { name: "Fetching…" })).toBeDisabled();
|
||||
rejectFetch(new Error("pricing unavailable"));
|
||||
await waitFor(() => expect(addToast).toHaveBeenCalledWith("pricing unavailable", "error"));
|
||||
await waitFor(() => expect(screen.getByRole("button", { name: "Fetch latest prices" })).not.toBeDisabled());
|
||||
});
|
||||
|
||||
it("fetch button calls the API and refreshes fetched pricing state", async () => {
|
||||
apiMock
|
||||
.mockResolvedValueOnce({ count: 1, fetchedAt: "2026-06-22T00:00:00.000Z", source: "litellm" })
|
||||
.mockResolvedValueOnce({
|
||||
modelPricingFetchedAt: "2026-06-22T00:00:00.000Z",
|
||||
modelPricingSource: "litellm",
|
||||
modelPricingOverrides: {
|
||||
"openai:gpt-test": {
|
||||
inputPer1M: 1,
|
||||
outputPer1M: 2,
|
||||
cacheReadPer1M: 1,
|
||||
cacheWritePer1M: 1,
|
||||
source: "litellm/model_prices_and_context_window.json",
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
render(<Harness initial={{} as SettingsFormState} />);
|
||||
fireEvent.click(screen.getByRole("button", { name: "Fetch latest prices" }));
|
||||
|
||||
await waitFor(() => expect(apiMock).toHaveBeenCalledWith(
|
||||
"/command-center/pricing/fetch?projectId=proj-a",
|
||||
{ method: "POST" },
|
||||
));
|
||||
await waitFor(() => expect(screen.getByText("openai:gpt-test")).toBeInTheDocument());
|
||||
expect(apiMock).toHaveBeenCalledTimes(2);
|
||||
expect(screen.getByText(/Prices as of/)).toBeInTheDocument();
|
||||
expect(screen.getByText(/litellm/)).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,193 @@
|
||||
import { useMemo, useState } from "react";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import type { ModelPricing, ModelPricingOverrides } from "@fusion/core";
|
||||
import { api } from "../../../api";
|
||||
import type { ToastType } from "../../../hooks/useToast";
|
||||
import type { SetSettingsForm, SettingsFormState } from "./context";
|
||||
import "./ModelPricingSection.css";
|
||||
|
||||
interface PricingFetchResponse {
|
||||
count: number;
|
||||
fetchedAt: string;
|
||||
source: string;
|
||||
}
|
||||
|
||||
interface ModelPricingSectionProps {
|
||||
form: SettingsFormState;
|
||||
setForm: SetSettingsForm;
|
||||
addToast: (message: string, type?: ToastType) => void;
|
||||
projectId?: string;
|
||||
}
|
||||
|
||||
interface PricingDraft {
|
||||
key: string;
|
||||
inputPer1M: number;
|
||||
outputPer1M: number;
|
||||
cacheReadPer1M: number;
|
||||
cacheWritePer1M: number;
|
||||
source: string;
|
||||
}
|
||||
|
||||
function pricingToDraft(key: string, pricing: ModelPricing): PricingDraft {
|
||||
return { key, ...pricing };
|
||||
}
|
||||
|
||||
function normalizePricingKey(value: string): string {
|
||||
return value.trim().toLowerCase();
|
||||
}
|
||||
|
||||
function parseRate(value: string): number {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : 0;
|
||||
}
|
||||
|
||||
function pricingPath(projectId?: string): string {
|
||||
return projectId
|
||||
? `/command-center/pricing/fetch?projectId=${encodeURIComponent(projectId)}`
|
||||
: "/command-center/pricing/fetch";
|
||||
}
|
||||
|
||||
function setOverrides(setForm: SetSettingsForm, overrides: ModelPricingOverrides): void {
|
||||
setForm((current) => ({
|
||||
...current,
|
||||
modelPricingOverrides: overrides,
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* FNXC:Settings 2026-06-22-00:00:
|
||||
* Global Models needs an editable model-pricing override table plus a one-click LiteLLM refresh. Edits flow through the existing Settings save path, while fetch persists immediately through the Command Center pricing route and then refreshes this form from global settings.
|
||||
*/
|
||||
export function ModelPricingSection({ form, setForm, addToast, projectId }: ModelPricingSectionProps) {
|
||||
const { t } = useTranslation("app");
|
||||
const [draft, setDraft] = useState<PricingDraft>({
|
||||
key: "",
|
||||
inputPer1M: 0,
|
||||
outputPer1M: 0,
|
||||
cacheReadPer1M: 0,
|
||||
cacheWritePer1M: 0,
|
||||
source: "manual",
|
||||
});
|
||||
const [fetching, setFetching] = useState(false);
|
||||
|
||||
const rows = useMemo(
|
||||
() => Object.entries(form.modelPricingOverrides ?? {}).sort(([a], [b]) => a.localeCompare(b)),
|
||||
[form.modelPricingOverrides],
|
||||
);
|
||||
|
||||
const updateRow = (key: string, patch: Partial<ModelPricing>) => {
|
||||
const current = form.modelPricingOverrides ?? {};
|
||||
const existing = current[key];
|
||||
if (!existing) return;
|
||||
setOverrides(setForm, {
|
||||
...current,
|
||||
[key]: { ...existing, ...patch },
|
||||
});
|
||||
};
|
||||
|
||||
const deleteRow = (key: string) => {
|
||||
const next = { ...(form.modelPricingOverrides ?? {}) };
|
||||
delete next[key];
|
||||
setOverrides(setForm, next);
|
||||
};
|
||||
|
||||
const addRow = () => {
|
||||
const key = normalizePricingKey(draft.key);
|
||||
if (!key || !key.includes(":")) {
|
||||
addToast(t("settings.modelPricing.invalidKey", "Use a provider:model key before adding a pricing row."), "error");
|
||||
return;
|
||||
}
|
||||
setOverrides(setForm, {
|
||||
...(form.modelPricingOverrides ?? {}),
|
||||
[key]: {
|
||||
inputPer1M: draft.inputPer1M,
|
||||
outputPer1M: draft.outputPer1M,
|
||||
cacheReadPer1M: draft.cacheReadPer1M,
|
||||
cacheWritePer1M: draft.cacheWritePer1M,
|
||||
source: draft.source || "manual",
|
||||
},
|
||||
});
|
||||
setDraft({ key: "", inputPer1M: 0, outputPer1M: 0, cacheReadPer1M: 0, cacheWritePer1M: 0, source: "manual" });
|
||||
};
|
||||
|
||||
const fetchLatestPrices = async () => {
|
||||
setFetching(true);
|
||||
try {
|
||||
const result = await api<PricingFetchResponse>(pricingPath(projectId), { method: "POST" });
|
||||
const settings = await api<Pick<SettingsFormState, "modelPricingOverrides" | "modelPricingFetchedAt" | "modelPricingSource">>("/settings/global");
|
||||
setForm((current) => ({
|
||||
...current,
|
||||
modelPricingOverrides: settings.modelPricingOverrides ?? current.modelPricingOverrides,
|
||||
modelPricingFetchedAt: settings.modelPricingFetchedAt ?? result.fetchedAt,
|
||||
modelPricingSource: settings.modelPricingSource ?? result.source,
|
||||
}));
|
||||
addToast(t("settings.modelPricing.fetchSuccess", "Fetched {{count}} model prices.", { count: result.count }), "success");
|
||||
} catch (error) {
|
||||
addToast(error instanceof Error ? error.message : t("settings.modelPricing.fetchFailed", "Failed to fetch latest model prices."), "error");
|
||||
} finally {
|
||||
setFetching(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<section className="model-pricing-section" aria-label={t("settings.modelPricing.title", "Model pricing overrides")}>
|
||||
<div className="model-pricing-section__header">
|
||||
<div>
|
||||
<h4 className="settings-section-heading settings-section-heading--spaced">{t("settings.modelPricing.title", "Model Pricing")}</h4>
|
||||
<p className="settings-description">
|
||||
{t("settings.modelPricing.description", "Override per-1M token rates used by Command Center cost estimates. Overrides win over the built-in baseline; unlisted models still use the baseline.")}
|
||||
</p>
|
||||
<p className="settings-muted model-pricing-section__meta">
|
||||
{form.modelPricingFetchedAt
|
||||
? t("settings.modelPricing.pricesAsOf", "Prices as of {{date}}", { date: new Date(form.modelPricingFetchedAt).toLocaleString() })
|
||||
: t("settings.modelPricing.noFetchYet", "No fetched pricing snapshot yet.")}
|
||||
{form.modelPricingSource ? ` · ${form.modelPricingSource}` : ""}
|
||||
</p>
|
||||
</div>
|
||||
<button type="button" className="btn btn-sm" onClick={() => void fetchLatestPrices()} disabled={fetching}>
|
||||
{fetching ? t("settings.modelPricing.fetching", "Fetching…") : t("settings.modelPricing.fetchLatest", "Fetch latest prices")}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="model-pricing-table" role="table" aria-label={t("settings.modelPricing.overrides", "Model pricing overrides")}>
|
||||
<div className="model-pricing-row model-pricing-row--head" role="row">
|
||||
<span role="columnheader">{t("settings.modelPricing.modelKey", "provider:model")}</span>
|
||||
<span role="columnheader">{t("settings.modelPricing.input", "Input / 1M")}</span>
|
||||
<span role="columnheader">{t("settings.modelPricing.output", "Output / 1M")}</span>
|
||||
<span role="columnheader">{t("settings.modelPricing.cacheRead", "Cache read / 1M")}</span>
|
||||
<span role="columnheader">{t("settings.modelPricing.cacheWrite", "Cache write / 1M")}</span>
|
||||
<span role="columnheader">{t("settings.modelPricing.source", "Source")}</span>
|
||||
<span role="columnheader">{t("settings.modelPricing.actions", "Actions")}</span>
|
||||
</div>
|
||||
{rows.length === 0 ? (
|
||||
<div className="settings-empty-state model-pricing-empty" role="row">
|
||||
{t("settings.modelPricing.empty", "No model pricing overrides yet. Add one manually or fetch the latest LiteLLM prices.")}
|
||||
</div>
|
||||
) : rows.map(([key, pricing]) => {
|
||||
const row = pricingToDraft(key, pricing);
|
||||
return (
|
||||
<div className="model-pricing-row" role="row" key={key}>
|
||||
<code className="model-pricing-key" role="cell">{row.key}</code>
|
||||
<input aria-label={`${key} input per 1M`} className="input" type="number" step="any" value={row.inputPer1M} onChange={(event) => updateRow(key, { inputPer1M: parseRate(event.target.value) })} />
|
||||
<input aria-label={`${key} output per 1M`} className="input" type="number" step="any" value={row.outputPer1M} onChange={(event) => updateRow(key, { outputPer1M: parseRate(event.target.value) })} />
|
||||
<input aria-label={`${key} cache read per 1M`} className="input" type="number" step="any" value={row.cacheReadPer1M} onChange={(event) => updateRow(key, { cacheReadPer1M: parseRate(event.target.value) })} />
|
||||
<input aria-label={`${key} cache write per 1M`} className="input" type="number" step="any" value={row.cacheWritePer1M} onChange={(event) => updateRow(key, { cacheWritePer1M: parseRate(event.target.value) })} />
|
||||
<input aria-label={`${key} source`} className="input" value={row.source} onChange={(event) => updateRow(key, { source: event.target.value })} />
|
||||
<button type="button" className="btn btn-ghost btn-sm" onClick={() => deleteRow(key)}>{t("settings.modelPricing.delete", "Delete")}</button>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
<div className="model-pricing-row model-pricing-row--add" role="row">
|
||||
<input aria-label={t("settings.modelPricing.newKey", "New provider:model key")} className="input" placeholder="openai:gpt-4o" value={draft.key} onChange={(event) => setDraft((current) => ({ ...current, key: event.target.value }))} />
|
||||
<input aria-label={t("settings.modelPricing.newInput", "New input rate")} className="input" type="number" step="any" value={draft.inputPer1M} onChange={(event) => setDraft((current) => ({ ...current, inputPer1M: parseRate(event.target.value) }))} />
|
||||
<input aria-label={t("settings.modelPricing.newOutput", "New output rate")} className="input" type="number" step="any" value={draft.outputPer1M} onChange={(event) => setDraft((current) => ({ ...current, outputPer1M: parseRate(event.target.value) }))} />
|
||||
<input aria-label={t("settings.modelPricing.newCacheRead", "New cache read rate")} className="input" type="number" step="any" value={draft.cacheReadPer1M} onChange={(event) => setDraft((current) => ({ ...current, cacheReadPer1M: parseRate(event.target.value) }))} />
|
||||
<input aria-label={t("settings.modelPricing.newCacheWrite", "New cache write rate")} className="input" type="number" step="any" value={draft.cacheWritePer1M} onChange={(event) => setDraft((current) => ({ ...current, cacheWritePer1M: parseRate(event.target.value) }))} />
|
||||
<input aria-label={t("settings.modelPricing.newSource", "New source")} className="input" value={draft.source} onChange={(event) => setDraft((current) => ({ ...current, source: event.target.value }))} />
|
||||
<button type="button" className="btn btn-sm" onClick={addRow}>{t("settings.modelPricing.addRow", "Add row")}</button>
|
||||
</div>
|
||||
</div>
|
||||
<small>{t("settings.modelPricing.saveHint", "Manual edits are saved with the rest of Global settings.")}</small>
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -8,8 +8,8 @@ import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { EventEmitter } from "node:events";
|
||||
|
||||
import { Database, emitUsageEvent } from "@fusion/core";
|
||||
import type { TaskStore } from "@fusion/core";
|
||||
import { Database, emitUsageEvent, LITELLM_PRICING_SOURCE_URL } from "@fusion/core";
|
||||
import type { GlobalSettings, TaskStore } from "@fusion/core";
|
||||
import { request } from "../test-request.js";
|
||||
import { ApiError } from "../api-error.js";
|
||||
import {
|
||||
@@ -21,6 +21,14 @@ import {
|
||||
} from "../routes/register-command-center-routes.js";
|
||||
import type { ApiRoutesContext } from "../routes/types.js";
|
||||
|
||||
const { mockInvalidateAllGlobalSettingsCaches } = vi.hoisted(() => ({
|
||||
mockInvalidateAllGlobalSettingsCaches: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("../project-store-resolver.js", () => ({
|
||||
invalidateAllGlobalSettingsCaches: mockInvalidateAllGlobalSettingsCaches,
|
||||
}));
|
||||
|
||||
/** Seed a temp DB with a token-bearing task and a tool-call usage event. */
|
||||
function seedDb(db: Database, opts: { taskId: string; model: string; tokens: number }): void {
|
||||
db.prepare(
|
||||
@@ -197,9 +205,27 @@ function buildApp(stores: Record<string, TaskStore>, fallback: TaskStore) {
|
||||
}
|
||||
|
||||
/** A minimal TaskStore exposing only the methods Command Center routes use. */
|
||||
function storeFor(db: Database, overrides: Partial<TaskStore> = {}): TaskStore {
|
||||
function storeFor(
|
||||
db: Database,
|
||||
overrides: Partial<TaskStore> = {},
|
||||
globalSettings: Partial<GlobalSettings> = {},
|
||||
): TaskStore {
|
||||
const store = new EventEmitter() as unknown as TaskStore & { getDatabase(): Database };
|
||||
const settings = {
|
||||
modelPricingOverrides: undefined,
|
||||
modelPricingFetchedAt: undefined,
|
||||
modelPricingSource: undefined,
|
||||
...globalSettings,
|
||||
} as GlobalSettings;
|
||||
store.getDatabase = () => db;
|
||||
store.getGlobalSettingsStore = () => ({
|
||||
getSettings: async () => settings,
|
||||
invalidateCache: vi.fn(),
|
||||
} as unknown as ReturnType<TaskStore["getGlobalSettingsStore"]>);
|
||||
store.updateGlobalSettings = vi.fn(async (patch: Partial<GlobalSettings>) => {
|
||||
Object.assign(settings, patch);
|
||||
return settings;
|
||||
}) as TaskStore["updateGlobalSettings"];
|
||||
Object.assign(store, overrides);
|
||||
return store;
|
||||
}
|
||||
@@ -227,6 +253,8 @@ describe("register-command-center-routes", () => {
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
mockInvalidateAllGlobalSettingsCaches.mockClear();
|
||||
dbA.close();
|
||||
dbB.close();
|
||||
rmSync(tmpDir, { recursive: true, force: true });
|
||||
@@ -263,6 +291,119 @@ describe("register-command-center-routes", () => {
|
||||
expect(body.series?.[0]).toHaveProperty("cost");
|
||||
});
|
||||
|
||||
it("token analytics applies persisted pricing overrides", async () => {
|
||||
const storeA = storeFor(dbA, {}, {
|
||||
modelPricingOverrides: {
|
||||
"anthropic:claude-sonnet-4-5": {
|
||||
inputPer1M: 1_000_000,
|
||||
outputPer1M: 1_000_000,
|
||||
cacheReadPer1M: 1_000_000,
|
||||
cacheWritePer1M: 1_000_000,
|
||||
source: "manual",
|
||||
},
|
||||
},
|
||||
});
|
||||
app = buildApp({ "proj-a": storeA }, storeA);
|
||||
|
||||
const res = await request(
|
||||
app,
|
||||
"GET",
|
||||
"/api/command-center/tokens?from=2026-02-01T00:00:00.000Z&to=2026-04-01T00:00:00.000Z&projectId=proj-a",
|
||||
);
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
expect((res.body as { cost: { usd: number } }).cost.usd).toBeCloseTo(200, 2);
|
||||
});
|
||||
|
||||
it("fetches latest pricing and persists merged global overrides", async () => {
|
||||
const storeA = storeFor(dbA, {}, {
|
||||
modelPricingOverrides: {
|
||||
"manual:custom": {
|
||||
inputPer1M: 9,
|
||||
outputPer1M: 9,
|
||||
cacheReadPer1M: 9,
|
||||
cacheWritePer1M: 9,
|
||||
source: "manual",
|
||||
},
|
||||
},
|
||||
});
|
||||
app = buildApp({ "proj-a": storeA }, storeA);
|
||||
vi.spyOn(globalThis, "fetch").mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
"gpt-test": {
|
||||
litellm_provider: "openai",
|
||||
mode: "chat",
|
||||
input_cost_per_token: 0.000001,
|
||||
output_cost_per_token: 0.000002,
|
||||
},
|
||||
}),
|
||||
text: async () => "",
|
||||
} as Response);
|
||||
|
||||
const res = await request(app, "POST", "/api/command-center/pricing/fetch?projectId=proj-a");
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body).toMatchObject({ count: 1, source: LITELLM_PRICING_SOURCE_URL });
|
||||
expect((res.body as { fetchedAt: string }).fetchedAt).toMatch(/^\d{4}-\d{2}-\d{2}T/);
|
||||
expect(storeA.updateGlobalSettings).toHaveBeenCalledWith(expect.objectContaining({
|
||||
modelPricingFetchedAt: expect.any(String),
|
||||
modelPricingSource: LITELLM_PRICING_SOURCE_URL,
|
||||
modelPricingOverrides: expect.objectContaining({
|
||||
"manual:custom": expect.objectContaining({ source: "manual" }),
|
||||
"openai:gpt-test": expect.objectContaining({ inputPer1M: 1, outputPer1M: 2 }),
|
||||
}),
|
||||
}));
|
||||
expect(mockInvalidateAllGlobalSettingsCaches).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("pricing fetch failures preserve existing overrides", async () => {
|
||||
const existing = {
|
||||
"anthropic:claude-sonnet-4-5": {
|
||||
inputPer1M: 99,
|
||||
outputPer1M: 99,
|
||||
cacheReadPer1M: 99,
|
||||
cacheWritePer1M: 99,
|
||||
source: "manual",
|
||||
},
|
||||
};
|
||||
const storeA = storeFor(dbA, {}, { modelPricingOverrides: existing });
|
||||
app = buildApp({ "proj-a": storeA }, storeA);
|
||||
vi.spyOn(globalThis, "fetch").mockRejectedValue(new Error("network down"));
|
||||
|
||||
const res = await request(app, "POST", "/api/command-center/pricing/fetch?projectId=proj-a");
|
||||
|
||||
expect(res.status).toBe(500);
|
||||
expect(storeA.updateGlobalSettings).not.toHaveBeenCalled();
|
||||
expect((await storeA.getGlobalSettingsStore().getSettings()).modelPricingOverrides).toEqual(existing);
|
||||
expect(mockInvalidateAllGlobalSettingsCaches).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("pricing fetch rejects empty parsed data without clobbering overrides", async () => {
|
||||
const existing = {
|
||||
"anthropic:claude-sonnet-4-5": {
|
||||
inputPer1M: 99,
|
||||
outputPer1M: 99,
|
||||
cacheReadPer1M: 99,
|
||||
cacheWritePer1M: 99,
|
||||
source: "manual",
|
||||
},
|
||||
};
|
||||
const storeA = storeFor(dbA, {}, { modelPricingOverrides: existing });
|
||||
app = buildApp({ "proj-a": storeA }, storeA);
|
||||
vi.spyOn(globalThis, "fetch").mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => ({ sample_spec: {}, "embedding-test": { litellm_provider: "openai", mode: "embedding" } }),
|
||||
text: async () => "",
|
||||
} as Response);
|
||||
|
||||
const res = await request(app, "POST", "/api/command-center/pricing/fetch?projectId=proj-a");
|
||||
|
||||
expect(res.status).toBe(502);
|
||||
expect(storeA.updateGlobalSettings).not.toHaveBeenCalled();
|
||||
expect((await storeA.getGlobalSettingsStore().getSettings()).modelPricingOverrides).toEqual(existing);
|
||||
});
|
||||
|
||||
it("ignores invalid token granularity rather than erroring", async () => {
|
||||
const res = await request(
|
||||
app,
|
||||
@@ -303,6 +444,31 @@ describe("register-command-center-routes", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("team analytics applies persisted pricing overrides", async () => {
|
||||
seedTeamMetrics(dbA, { agentId: "agent-route-a", name: "Route Alpha", tokens: 100, taskId: "FN-A-team-override" });
|
||||
const storeA = storeFor(dbA, {}, {
|
||||
modelPricingOverrides: {
|
||||
"anthropic:claude-sonnet-4-5": {
|
||||
inputPer1M: 1_000_000,
|
||||
outputPer1M: 1_000_000,
|
||||
cacheReadPer1M: 1_000_000,
|
||||
cacheWritePer1M: 1_000_000,
|
||||
source: "manual",
|
||||
},
|
||||
},
|
||||
});
|
||||
app = buildApp({ "proj-a": storeA }, storeA);
|
||||
|
||||
const res = await request(
|
||||
app,
|
||||
"GET",
|
||||
"/api/command-center/team?from=2026-02-01T00:00:00.000Z&to=2026-04-01T00:00:00.000Z&projectId=proj-a",
|
||||
);
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
expect((res.body as { totals: { cost: { usd: number } } }).totals.cost.usd).toBeCloseTo(200, 2);
|
||||
});
|
||||
|
||||
it("returns the tools / activity / productivity aggregator shapes", async () => {
|
||||
const range = "from=2026-02-01T00:00:00.000Z&to=2026-04-01T00:00:00.000Z";
|
||||
seedAgentRun(dbA, { id: "run-a1", agentId: "agent-route", startedAt: "2026-03-02T00:00:00.000Z", status: "active" });
|
||||
|
||||
@@ -8,6 +8,8 @@ import {
|
||||
aggregateGithubIssueAnalytics,
|
||||
aggregateSignalsAnalytics,
|
||||
composeLiveSnapshot,
|
||||
LITELLM_PRICING_SOURCE_URL,
|
||||
parseLiteLLMPricing,
|
||||
type TokenGroupBy,
|
||||
type TokenTimeGranularity,
|
||||
} from "@fusion/core";
|
||||
@@ -22,6 +24,7 @@ import {
|
||||
githubIssueAnalyticsToTable,
|
||||
type CsvTable,
|
||||
} from "../command-center-csv.js";
|
||||
import { invalidateAllGlobalSettingsCaches } from "../project-store-resolver.js";
|
||||
import type { ApiRouteRegistrar } from "./types.js";
|
||||
|
||||
/**
|
||||
@@ -138,6 +141,22 @@ function sendCsv(res: Response, filename: string, table: CsvTable): void {
|
||||
res.send(serializeCsv(table));
|
||||
}
|
||||
|
||||
const PRICING_FETCH_TIMEOUT_MS = 10_000;
|
||||
|
||||
async function fetchLatestLiteLLMPricing(): Promise<unknown> {
|
||||
const response = await fetch(LITELLM_PRICING_SOURCE_URL, {
|
||||
signal: AbortSignal.timeout(PRICING_FETCH_TIMEOUT_MS),
|
||||
});
|
||||
if (!response.ok) {
|
||||
const body = await response.text().catch(() => "");
|
||||
throw new ApiError(
|
||||
502,
|
||||
`Failed to fetch pricing source: ${response.status} ${response.statusText}${body ? `: ${body.slice(0, 200)}` : ""}`,
|
||||
);
|
||||
}
|
||||
return response.json() as Promise<unknown>;
|
||||
}
|
||||
|
||||
export const registerCommandCenterRoutes: ApiRouteRegistrar = (ctx) => {
|
||||
const { router, getScopedStore, rethrowAsApiError } = ctx;
|
||||
|
||||
@@ -152,12 +171,14 @@ export const registerCommandCenterRoutes: ApiRouteRegistrar = (ctx) => {
|
||||
const range = resolveRange(req.query);
|
||||
const groupBy = resolveGroupBy(req.query);
|
||||
const granularity = resolveTokenGranularity(req.query);
|
||||
const settings = await store.getGlobalSettingsStore().getSettings();
|
||||
const result = aggregateTokenAnalytics(store.getDatabase(), {
|
||||
from: range.from,
|
||||
to: range.to,
|
||||
groupBy,
|
||||
granularity,
|
||||
now: Date.now(),
|
||||
pricingOverrides: settings.modelPricingOverrides,
|
||||
});
|
||||
if (wantsCsv(req.query)) {
|
||||
sendCsv(res, "command-center-tokens.csv", tokenAnalyticsToTable(result));
|
||||
@@ -170,6 +191,41 @@ export const registerCommandCenterRoutes: ApiRouteRegistrar = (ctx) => {
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* POST /api/command-center/pricing/fetch
|
||||
* Fetch + persist user-editable model-pricing overrides from LiteLLM.
|
||||
*
|
||||
* FNXC:CommandCenter 2026-06-22-00:00:
|
||||
* Operators need a one-click refresh from the pinned LiteLLM JSON dataset without adding HTTP to core pricing. Preserve existing overrides on fetch/parse failures and invalidate global settings caches after a successful write so Command Center cost reads use the refreshed rates immediately.
|
||||
*/
|
||||
router.post("/command-center/pricing/fetch", async (req, res) => {
|
||||
try {
|
||||
const store = await getScopedStore(req);
|
||||
const json = await fetchLatestLiteLLMPricing();
|
||||
const parsed = parseLiteLLMPricing(json);
|
||||
if (parsed.count === 0) {
|
||||
throw new ApiError(502, "No chat-mode pricing entries found in fetched LiteLLM data");
|
||||
}
|
||||
|
||||
const settings = await store.getGlobalSettingsStore().getSettings();
|
||||
const fetchedAt = new Date().toISOString();
|
||||
await store.updateGlobalSettings({
|
||||
modelPricingOverrides: {
|
||||
...(settings.modelPricingOverrides ?? {}),
|
||||
...parsed.overrides,
|
||||
},
|
||||
modelPricingFetchedAt: fetchedAt,
|
||||
modelPricingSource: LITELLM_PRICING_SOURCE_URL,
|
||||
});
|
||||
invalidateAllGlobalSettingsCaches();
|
||||
|
||||
res.json({ count: parsed.count, fetchedAt, source: LITELLM_PRICING_SOURCE_URL });
|
||||
} catch (err: unknown) {
|
||||
if (err instanceof ApiError) throw err;
|
||||
rethrowAsApiError(err, "Failed to fetch model pricing");
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* GET /api/command-center/tools
|
||||
* Tool-usage counts + autonomy ratio (U2) over a date range.
|
||||
@@ -274,10 +330,12 @@ export const registerCommandCenterRoutes: ApiRouteRegistrar = (ctx) => {
|
||||
try {
|
||||
const store = await getScopedStore(req);
|
||||
const range = resolveRange(req.query);
|
||||
const settings = await store.getGlobalSettingsStore().getSettings();
|
||||
const result = aggregateTeamAnalytics(store.getDatabase(), {
|
||||
from: range.from,
|
||||
to: range.to,
|
||||
now: Date.now(),
|
||||
pricingOverrides: settings.modelPricingOverrides,
|
||||
});
|
||||
res.json(result);
|
||||
} catch (err: unknown) {
|
||||
|
||||
Reference in New Issue
Block a user