diff --git a/.changeset/fn-6669-resolved-model-cost.md b/.changeset/fn-6669-resolved-model-cost.md new file mode 100644 index 0000000000..5ee4ae39b2 --- /dev/null +++ b/.changeset/fn-6669-resolved-model-cost.md @@ -0,0 +1,5 @@ +--- +"@runfusion/fusion": patch +--- + +Command Center token-cost analytics now price resolved-via-settings task usage from the actually-used model snapshot, with legacy own-model fallback, instead of showing those costs as unavailable. diff --git a/docs/dashboard-guide.md b/docs/dashboard-guide.md index f8919cd461..d5b053bd34 100644 --- a/docs/dashboard-guide.md +++ b/docs/dashboard-guide.md @@ -664,7 +664,7 @@ Navigation: Features: - Global date-range picker in the header scopes the analytics tabs; **Mission Control** remains live rather than historical. - **Overview** summarizes token usage/cost, autonomy, active nodes, agent runs, tasks done, model breadth, and open signals, and includes the SDLC throughput funnel for the selected range. Its token total and Live activity snapshot token metric refresh on a bounded live cadence and animate number changes while preserving reduced-motion preferences. The Live activity snapshot also shows the current board-state count for tasks in progress, independent of the selected analytics date range. Overview includes a graph-rich software-factory snapshot with tokens-by-model, tool-category, and daily activity trend charts that reuse the already-loaded tokens, tools, and activity analytics. The chart reveal/glow accents are decorative and disabled when reduced-motion preferences are active. The SDLC completion rate is shown as a radial gauge and is calculated as cohort conversion from in-range triage entrants, so the rate is capped at 100% even when older tasks finish during the range. -- **Tokens** breaks down token totals, estimated cost, tasks, and per-model usage. Per-model and per-provider breakdowns use the task's analytics-only actually-used model snapshot when available, so usage from settings-resolved runs appears under the real runtime model instead of `(unknown)` without changing future model resolution. It also includes a live token-usage-over-time chart backed by per-task token timestamps; use the granularity control to switch the chart between hourly, daily, and weekly buckets. The token total and chart poll on a bounded cadence, keep the previous data visible during refresh, animate decorative count/bar transitions, and disable those animations for reduced-motion users. +- **Tokens** breaks down token totals, estimated cost, tasks, and per-model usage. Per-model and per-provider breakdowns use the task's analytics-only actually-used model snapshot when available, so usage from settings-resolved runs appears under the real runtime model instead of `(unknown)` without changing future model resolution; estimated cost uses the same snapshot-first, legacy-fallback model identity so those resolved runs price normally when the model is in the pricing table. It also includes a live token-usage-over-time chart backed by per-task token timestamps; use the granularity control to switch the chart between hourly, daily, and weekly buckets. The token total and chart poll on a bounded cadence, keep the previous data visible during refresh, animate decorative count/bar transitions, and disable those animations for reduced-motion users. - **Tools** shows autonomy ratio, tool-call volume, intervention counts, sessions, and tool categories. - **Activity** tracks sessions, messages, active nodes, active agents, agent heartbeat runs, and stickiness. Agent-run sheets show total, active, completed, and failed runs for the selected range, and the Agent runs/day sparkline trends runs by `agentRuns.startedAt`. The area also renders live animated line charts for messages/day, active agents/day, active nodes/day, and combined throughput/day (`messages + active agents + active nodes`). These charts reuse the existing activity analytics endpoint, refresh on a bounded 15-second cadence while mounted, keep the previous data visible during refreshes, and disable decorative draw-on motion for reduced-motion users. - **Productivity** separates outcome counters (commits and pull requests) from volume proxies such as modified files, lines changed, and files by language. diff --git a/docs/storage.md b/docs/storage.md index e173b57682..6da4f12538 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -388,7 +388,7 @@ FN-5240/FN-5241/FN-5242 establish the handoff invariant: the only legal executor The `tasks.githubTracking` JSON column stores per-task GitHub tracking state (`enabled`, optional `repoOverride`, linked issue metadata, and `unlinkedAt`). It is additive and default-off; imported-source issue metadata remains in `issueInfo` / `sourceIssue`. Behavior wiring (issue creation/lifecycle sync and UI surfacing) lands in FN-3870/FN-3873/FN-3874. -The `tasks.tokenUsage*` columns store cumulative per-task token usage for analytics. `tokenUsageModelProvider` and `tokenUsageModelId` are analytics-only snapshots of the actually-used runtime model recorded when usage is accumulated; they let Command Center group resolved-via-settings usage by provider/model without writing the task-level `modelProvider` / `modelId` own-model override fields that control future model resolution. +The `tasks.tokenUsage*` columns store cumulative per-task token usage for analytics. `tokenUsageModelProvider` and `tokenUsageModelId` are analytics-only snapshots of the actually-used runtime model recorded when usage is accumulated; they let Command Center group and price resolved-via-settings usage by provider/model without writing the task-level `modelProvider` / `modelId` own-model override fields that control future model resolution. Cost attribution reads the snapshot first and falls back to the legacy own-model columns for pre-snapshot rows. | `config` | Single-row project configuration (`nextId`, settings payload, workflow step counters). | | `workflow_steps` | Workflow step definitions (`prompt`/`script`) with phase, template metadata, and model overrides. | | `activityLog` | Per-project activity/event log with timestamp/type/task indexes. | diff --git a/packages/core/src/__tests__/token-analytics.test.ts b/packages/core/src/__tests__/token-analytics.test.ts index ad2fbb9868..b4afd06bf9 100644 --- a/packages/core/src/__tests__/token-analytics.test.ts +++ b/packages/core/src/__tests__/token-analytics.test.ts @@ -5,6 +5,7 @@ import { join } from "node:path"; import { tmpdir } from "node:os"; import { Database } from "../db.js"; +import { costFor } from "../model-pricing.js"; import { aggregateTokenAnalytics } from "../token-analytics.js"; interface TaskSeed { @@ -268,6 +269,87 @@ describe("token-analytics", () => { expect(result.series?.[0].cost).toEqual({ usd: 12.5, unavailable: true, stale: false }); }); + it("prices resolved-model token usage costs from the usage snapshot across analytics surfaces", () => { + const usage = { inputTokens: 1_000_000, outputTokens: 1_000_000, cachedTokens: 0, cacheWriteTokens: 0 }; + const expected = costFor(usage, { provider: "openai", model: "gpt-4o" }); + expect(expected).toEqual({ usd: 12.5, unavailable: false, stale: false }); + + insertTask(db, { + id: "resolved", + ...usage, + totalTokens: 2_000_000, + lastUsedAt: "2026-03-01T00:00:00.000Z", + modelProvider: null, + modelId: null, + tokenUsageModelProvider: "openai", + tokenUsageModelId: "gpt-4o", + nodeId: "node-resolved", + agentId: "agent-resolved", + }); + + const byModel = aggregateTokenAnalytics(db, { groupBy: "model" }); + const modelGroup = byModel.groups.find((group) => group.key === "gpt-4o"); + expect(modelGroup?.cost).toEqual(expected); + expect(modelGroup?.cost.unavailable).toBe(false); + expect(byModel.cost).toEqual(expected); + + const byProvider = aggregateTokenAnalytics(db, { groupBy: "provider" }); + expect(byProvider.groups.find((group) => group.key === "openai")?.cost).toEqual(expected); + + const byNode = aggregateTokenAnalytics(db, { groupBy: "node" }); + expect(byNode.groups.find((group) => group.key === "node-resolved")?.cost).toEqual(expected); + + const byAgent = aggregateTokenAnalytics(db, { groupBy: "agent" }); + expect(byAgent.groups.find((group) => group.key === "agent-resolved")?.cost).toEqual(expected); + + const byDay = aggregateTokenAnalytics(db, { granularity: "day" }); + expect(byDay.series).toHaveLength(1); + expect(byDay.series?.[0].cost).toEqual(expected); + }); + + it("keeps token cost fallback and snapshot precedence guess-free", () => { + const usage = { inputTokens: 1_000_000, outputTokens: 1_000_000, cachedTokens: 0, cacheWriteTokens: 0 }; + const legacyExpected = costFor(usage, { provider: "openai", model: "gpt-4o-mini" }); + const snapshotExpected = costFor(usage, { provider: "openai", model: "gpt-4o" }); + expect(legacyExpected.usd).not.toBe(snapshotExpected.usd); + + insertTask(db, { + id: "legacy-priced", + ...usage, + totalTokens: 2_000_000, + lastUsedAt: "2026-03-01T00:00:00.000Z", + modelProvider: "openai", + modelId: "gpt-4o-mini", + }); + insertTask(db, { + id: "snapshot-wins", + ...usage, + totalTokens: 2_000_000, + lastUsedAt: "2026-03-02T00:00:00.000Z", + modelProvider: "openai", + modelId: "gpt-4o-mini", + tokenUsageModelProvider: "openai", + tokenUsageModelId: "gpt-4o", + }); + insertTask(db, { + id: "unpriced-snapshot", + inputTokens: 100, + totalTokens: 100, + lastUsedAt: "2026-03-03T00:00:00.000Z", + modelProvider: "openai", + modelId: "gpt-4o", + tokenUsageModelProvider: "unknown", + tokenUsageModelId: "mystery-model", + }); + + const result = aggregateTokenAnalytics(db, { groupBy: "model" }); + const groups = new Map(result.groups.map((group) => [group.key, group])); + + expect(groups.get("gpt-4o-mini")?.cost).toEqual(legacyExpected); + expect(groups.get("gpt-4o")?.cost).toEqual(snapshotExpected); + expect(groups.get("mystery-model")?.cost).toEqual({ usd: null, unavailable: true, stale: false }); + }); + it("returns an empty series for an empty requested range", () => { insertTask(db, { id: "t1", inputTokens: 100, totalTokens: 100, lastUsedAt: "2026-03-01T00:00:00.000Z", modelId: "model-A" }); diff --git a/packages/core/src/token-analytics.ts b/packages/core/src/token-analytics.ts index c073f3f187..f7ee561263 100644 --- a/packages/core/src/token-analytics.ts +++ b/packages/core/src/token-analytics.ts @@ -148,6 +148,10 @@ function emptyCostAccumulator(): CostAccumulator { } function addRowCost(acc: CostAccumulator, row: TaskTokenRow, now?: number): void { + /* + * FNXC:CommandCenter 2026-06-18-12:00: + * Token cost attribution must use the actually-used model snapshot first, then legacy own-model columns, matching groupKeyFor so resolved-via-settings tasks show priced Command Center costs instead of unavailable groups. + */ const result = costFor( { inputTokens: row.inputTokens ?? 0, @@ -155,7 +159,10 @@ function addRowCost(acc: CostAccumulator, row: TaskTokenRow, now?: number): void cachedTokens: row.cachedTokens ?? 0, cacheWriteTokens: row.cacheWriteTokens ?? 0, }, - { provider: row.modelProvider, model: row.modelId }, + { + provider: row.tokenUsageModelProvider ?? row.modelProvider, + model: row.tokenUsageModelId ?? row.modelId, + }, now, ); if (result.stale) acc.anyStale = true;