FN-6714: backfill commit-association diff stats
Backfill historical commit-association diff stats so Command Center LOC can use real persisted git data. - Add a core backfill API that inspects local git commits for associations with missing additions/deletions and supports dry-run reporting. - Expose an authenticated Command Center productivity backfill endpoint and route it through the legacy API shim. - Keep productivity LOC/hour sentinels safe for legacy payloads and document the backfill contract. - Cover the backfill path, auth gate, productivity analytics, and route registration with tests. Files changed: .changeset/fn-6714-command-center-loc-backfill.md | 5 + docs/architecture.md | 4 +- docs/storage.md | 4 +- ...mmit-association-diff-backfill.real-git.test.ts | 140 +++++++++++++++++++++ packages/core/src/index.ts | 1 + packages/core/src/store.ts | 76 ++++++++++- packages/core/src/types.ts | 9 ++ packages/dashboard/app/api/legacy.ts | 16 +++ .../command-center/areas/ProductivityArea.tsx | 8 +- .../register-command-center-routes.auth.test.ts | 48 ++++--- .../register-command-center-routes.test.ts | 49 +++++++- .../src/routes/register-command-center-routes.ts | 20 +++ 12 files changed, 354 insertions(+), 26 deletions(-) Fusion-Task-Id: FN-6714 Fusion-Task-Lineage: 657c7789-bbe4-4611-abca-127af05b0300
This commit is contained in:
gsxdsm
5
.changeset/fn-6714-command-center-loc-backfill.md
Normal file
5
.changeset/fn-6714-command-center-loc-backfill.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"@runfusion/fusion": minor
|
||||
---
|
||||
|
||||
Add an operator-triggered Command Center Productivity LOC backfill API and client for historical commit-association diff stats.
|
||||
@@ -861,7 +861,7 @@ Operator setup + troubleshooting guide: **[Remote Access runbook](./remote-acces
|
||||
Key server capabilities:
|
||||
- REST APIs for tasks, git, GitHub, agents, missions, planning, automations/routines, settings
|
||||
- System stats snapshot and vitest process controls APIs (`GET /api/system-stats`, `POST /api/kill-vitest`) exposing dashboard process/system telemetry (including app CPU percentage and host memory rendered as numeric values, radial gauges, and trend sparklines in the Command Center System area), task/agent aggregates, and manual vitest process termination. Host-memory usage is derived from shared OS-available memory (`process.availableMemory()` with an unreliable `freemem` fallback) rather than raw free pages so macOS inactive/cache memory is not counted as used.
|
||||
- Command Center analytics APIs (`GET /api/command-center/tokens`, `/tools`, `/activity`, `/productivity`, `/team`, `/github`, `/signals`, `/plugin-activations`, `/live`) are project-scoped dashboard routes. `/productivity` reads Lines changed from nullable `task_commit_associations.additions`/`deletions` merge-time diff stats, derives estimated `hoursSaved` from that LOC via the exported `HUMAN_LINES_PER_HOUR` rate, and keeps the unavailable sentinel for both fields when no in-range association has stats. Its `taskDuration` payload aggregates done tasks whose `executionCompletedAt` falls in the selected range, using positive `tasks.cumulativeActiveMs` values for completed count, average, median, p90, and total active execution time; missing qualifying durations remain unavailable rather than zero. `/signals` aggregates real local `incidents` rows for total/open/resolved counts, MTTR, and source/severity/status breakdowns and returns honest empty/unavailable sentinels instead of synthetic signal volume. `/plugin-activations` aggregates persisted plugin/extension load events for the selected range and returns unavailable when no rows exist instead of treating missing history as zero activations.
|
||||
- Command Center analytics APIs (`GET /api/command-center/tokens`, `/tools`, `/activity`, `/productivity`, `/team`, `/github`, `/signals`, `/plugin-activations`, `/live`) are project-scoped dashboard routes. `/productivity` reads Lines changed from nullable `task_commit_associations.additions`/`deletions` merge-time or backfilled diff stats, derives estimated `hoursSaved` from that LOC via the exported `HUMAN_LINES_PER_HOUR` rate, and keeps the unavailable sentinel for both fields when no in-range association has stats. `POST /api/command-center/productivity/backfill-loc` is the explicit operator-triggered, dry-run-defaulting local-git backfill for historical NULL stats; it is not run during dashboard rendering or analytics reads. Its `taskDuration` payload aggregates done tasks whose `executionCompletedAt` falls in the selected range, using positive `tasks.cumulativeActiveMs` values for completed count, average, median, p90, and total active execution time; missing qualifying durations remain unavailable rather than zero. `/signals` aggregates real local `incidents` rows for total/open/resolved counts, MTTR, and source/severity/status breakdowns and returns honest empty/unavailable sentinels instead of synthetic signal volume. `/plugin-activations` aggregates persisted plugin/extension load events for the selected range and returns unavailable when no rows exist instead of treating missing history as zero activations.
|
||||
<!-- FNXC:CommandCenter 2026-06-21-00:00: Maintainers need the pricing contract in architecture docs: MODEL_PRICING is hand-maintained, pricingAsOf changes with every rate edit, provider coverage includes OpenAI/Codex/Anthropic/Gemini, and Command Center never guesses or persists costs. -->
|
||||
- Model pricing & cost estimation: Command Center token cost is derived at read time by `packages/core/src/model-pricing.ts`, not fetched from providers at runtime and not persisted as billing truth. Maintainers update the hand-maintained `MODEL_PRICING` table in that file; keys are lowercased `${provider}:${model}` with a bare `:model` fallback for callers that only know the model id. Each entry stores USD per 1M tokens for input, output, cache-read, and cache-write plus a `source` citation. Bump `pricingAsOf` in the same change as any rate edit, because the dashboard surfaces it as the **prices as of** date and marks entries low-confidence after `PRICING_STALE_AFTER_MS` (approximately 180 days / two quarters) relative to that date. Unknown models resolve to `unavailable` rather than a guessed price. The table is curated from provider pricing pages for Anthropic, OpenAI including explicit `openai-codex:*` Codex ids, and Google Gemini; keep provider/model additions in that curated map rather than adding runtime pricing fetches.
|
||||
- Remote access APIs (`/api/remote/*`) for provider config, activation, tunnel lifecycle, status, token issuance, authenticated URL generation, and QR payload generation
|
||||
@@ -1468,7 +1468,7 @@ Dashboard session-diff route registration (`packages/dashboard/src/routes/regist
|
||||
- `legacy` = recovered via legacy task-id/subject matching
|
||||
- `ambiguous` = manual reconciliation where historical task-id attribution could be misleading
|
||||
|
||||
Commit associations also carry optional `additions`/`deletions` shortstat counts captured by merge paths. These nullable fields are the Command Center Productivity LOC source: analytics sum additions + deletions only when at least one in-range row has stats, derive estimated human hours saved as `round((additions + deletions) / HUMAN_LINES_PER_HOUR, 1)`, and preserve the `—` unavailable sentinel for both LOC and hours saved when all matching rows are `NULL` so unknown historical data is never rendered as `0`. The hours-saved field is a conservative estimate, not exact time tracking.
|
||||
Commit associations also carry optional `additions`/`deletions` shortstat counts captured by merge paths or filled later by the explicit `POST /api/command-center/productivity/backfill-loc` operator backfill. These nullable fields are the Command Center Productivity LOC source: analytics sum additions + deletions only when at least one in-range row has stats, derive estimated human hours saved as `round((additions + deletions) / HUMAN_LINES_PER_HOUR, 1)`, and preserve the `—` unavailable sentinel for both LOC and hours saved when all matching rows are `NULL` so unknown historical data is never rendered as `0`. The backfill only touches rows where both columns are `NULL`; malformed SHAs and commit objects unavailable in the local repo stay `NULL`, so partial historical coverage remains visible until a real local git object supplies stats. The hours-saved field is a conservative estimate, not exact time tracking.
|
||||
|
||||
Command Center Productivity task-duration stats use task rows, not commit rows: done tasks completed in range (`executionCompletedAt`) contribute when `cumulativeActiveMs > 0`. The aggregator computes completed count plus average, median, p90, and total active execution milliseconds; if no qualifying task exists, the duration metrics use the same unavailable `—` contract instead of reporting `0`.
|
||||
|
||||
|
||||
@@ -410,13 +410,13 @@ The `tasks.tokenUsage*` columns store cumulative per-task token usage for analyt
|
||||
|
||||
The nullable `tasks.tokenUsagePerModel` JSON column (migration 125) stores the per-task, per-runtime-model breakdown behind those cumulative totals. Each bucket records provider/model, token counts, and first/last use timestamps. Command Center model/provider analytics expand these buckets so multi-model tasks appear under every model they actually used; task-level totals, cost, time series, node grouping, and agent grouping still read the top-level aggregate so grand `nTasks` is not double-counted. Empty, missing, or malformed per-model JSON falls back to the legacy single-snapshot grouping path.
|
||||
|
||||
The `task_commit_associations.additions` and `task_commit_associations.deletions` columns (migration 123) store nullable merge-time git shortstat counts for the associated commit. Command Center Productivity uses `SUM(additions + deletions)` as the Lines changed source when at least one in-range association has non-null stats, then derives estimated `hoursSaved` as `round(loc / HUMAN_LINES_PER_HOUR, 1)`. `NULL` means stats were unknown or unavailable for that association, not zero; ranges with no non-null stats keep the unavailable `—` sentinel for both LOC and hours saved instead of reporting `0`.
|
||||
The `task_commit_associations.additions` and `task_commit_associations.deletions` columns (migration 123) store nullable merge-time git shortstat counts for the associated commit. Command Center Productivity uses `SUM(additions + deletions)` as the Lines changed source when at least one in-range association has non-null stats, then derives estimated `hoursSaved` as `round(loc / HUMAN_LINES_PER_HOUR, 1)`. `NULL` means stats were unknown or unavailable for that association, not zero; ranges with no non-null stats keep the unavailable `—` sentinel for both LOC and hours saved instead of reporting `0`. Historical rows created before diff-stat capture can be backfilled from local git with the explicit operator action `POST /api/command-center/productivity/backfill-loc` (dry-run by default). The backfill only updates rows where both columns are `NULL`; it validates commit SHAs before invoking git, leaves malformed or locally unavailable commit objects as `NULL`, and never overwrites already-populated stats.
|
||||
|
||||
The `tasks.cumulativeActiveMs` and `tasks.executionCompletedAt` columns are the Command Center Productivity task-duration source. Duration analytics select `column = 'done'` tasks completed in the requested range (`executionCompletedAt`) and include only positive `cumulativeActiveMs` values, then compute completed count, average, median, p90, and total active execution time. Missing, zero, or historical untracked duration values remain unavailable (`—`) rather than being serialized or rendered as `0`.
|
||||
| `config` | Single-row project configuration (`nextId`, settings payload, workflow step counters). |
|
||||
| `workflow_steps` | Workflow step definitions (`prompt`/`script`) with phase, template metadata, and model overrides. |
|
||||
| `activityLog` | Per-project activity/event log with timestamp/type/task indexes. |
|
||||
| `task_commit_associations` | Commit-to-task-lineage associations for canonical and legacy landed-commit attribution. Includes nullable `additions`/`deletions` diff-stat columns captured at merge time for Command Center Productivity LOC and derived estimated `hoursSaved`; `NULL` means stats unknown, not zero. |
|
||||
| `task_commit_associations` | Commit-to-task-lineage associations for canonical and legacy landed-commit attribution. Includes nullable `additions`/`deletions` diff-stat columns captured at merge time or by the explicit NULL-only local-git backfill for Command Center Productivity LOC and derived estimated `hoursSaved`; `NULL` means stats unknown, not zero. |
|
||||
| `archivedTasks` | Archived task snapshots (compact JSON payload + archive timestamp). |
|
||||
| `automations` | Scheduled automation definitions, run state, and run history. |
|
||||
| `agents` | Agent registry/state/task assignment metadata. |
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
import { execSync } from "node:child_process";
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
|
||||
import { TaskStore } from "../store.js";
|
||||
|
||||
function git(command: string, cwd: string): string {
|
||||
return execSync(command, { cwd, encoding: "utf8", stdio: ["pipe", "pipe", "pipe"] }).trim();
|
||||
}
|
||||
|
||||
function insertAssociation(
|
||||
store: TaskStore,
|
||||
input: {
|
||||
id: string;
|
||||
lineageId: string;
|
||||
sha: string;
|
||||
matchedBy?: string;
|
||||
additions?: number | null;
|
||||
deletions?: number | null;
|
||||
},
|
||||
): void {
|
||||
const authoredAt = "2026-06-19T00:00:00.000Z";
|
||||
(store as any).db.prepare(
|
||||
`INSERT INTO task_commit_associations
|
||||
(id, taskLineageId, taskIdSnapshot, commitSha, commitSubject, authoredAt,
|
||||
matchedBy, confidence, additions, deletions, createdAt, updatedAt)
|
||||
VALUES (?, ?, 'FN-6714', ?, 'subject', ?, ?, 'canonical', ?, ?, ?, ?)`,
|
||||
).run(
|
||||
input.id,
|
||||
input.lineageId,
|
||||
input.sha,
|
||||
authoredAt,
|
||||
input.matchedBy ?? "canonical-lineage-trailer",
|
||||
input.additions ?? null,
|
||||
input.deletions ?? null,
|
||||
authoredAt,
|
||||
authoredAt,
|
||||
);
|
||||
}
|
||||
|
||||
function readStats(store: TaskStore, id: string): { additions: number | null; deletions: number | null; updatedAt: string } {
|
||||
return (store as any).db.prepare(
|
||||
`SELECT additions, deletions, updatedAt FROM task_commit_associations WHERE id = ?`,
|
||||
).get(id) as { additions: number | null; deletions: number | null; updatedAt: string };
|
||||
}
|
||||
|
||||
/**
|
||||
* FNXC:CommandCenterProductivity 2026-06-21-00:00:
|
||||
* Historical task commit associations may predate LOC columns, so the backfill contract must be proven against real git shortstat output while preserving populated rows and treating invalid or unavailable SHAs as non-fatal.
|
||||
*/
|
||||
describe("TaskStore.backfillCommitAssociationDiffStats", () => {
|
||||
let rootDir: string;
|
||||
let globalDir: string;
|
||||
let store: TaskStore;
|
||||
|
||||
beforeEach(async () => {
|
||||
rootDir = mkdtempSync(join(tmpdir(), "fn-commit-diff-backfill-repo-"));
|
||||
globalDir = mkdtempSync(join(tmpdir(), "fn-commit-diff-backfill-global-"));
|
||||
git("git init --initial-branch=main", rootDir);
|
||||
git('git config user.name "Fusion Test"', rootDir);
|
||||
git('git config user.email "test@example.com"', rootDir);
|
||||
|
||||
store = new TaskStore(rootDir, globalDir);
|
||||
await store.init();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
store.close();
|
||||
rmSync(rootDir, { recursive: true, force: true });
|
||||
rmSync(globalDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("fills only NULL historical rows from local git and leaves unknown objects NULL", async () => {
|
||||
mkdirSync(join(rootDir, "src"), { recursive: true });
|
||||
writeFileSync(join(rootDir, "src", "added.txt"), "one\n");
|
||||
git("git add src/added.txt", rootDir);
|
||||
git('git commit -m "add one line"', rootDir);
|
||||
const addOnlySha = git("git rev-parse HEAD", rootDir);
|
||||
|
||||
writeFileSync(join(rootDir, "src", "changed.txt"), "one\ntwo\nthree\n");
|
||||
git("git add src/changed.txt", rootDir);
|
||||
git('git commit -m "add three lines"', rootDir);
|
||||
|
||||
writeFileSync(join(rootDir, "src", "changed.txt"), "one\n");
|
||||
git("git add src/changed.txt", rootDir);
|
||||
git('git commit -m "delete two lines"', rootDir);
|
||||
const deletionSha = git("git rev-parse HEAD", rootDir);
|
||||
|
||||
const unavailableSha = "abcdef1";
|
||||
const maliciousSha = "bad;touch should-not-exist";
|
||||
insertAssociation(store, { id: "null-add-1", lineageId: "lin-a", sha: addOnlySha });
|
||||
insertAssociation(store, { id: "null-add-2", lineageId: "lin-b", sha: addOnlySha, matchedBy: "legacy-subject" });
|
||||
insertAssociation(store, { id: "null-delete", lineageId: "lin-c", sha: deletionSha });
|
||||
insertAssociation(store, { id: "unavailable", lineageId: "lin-d", sha: unavailableSha });
|
||||
insertAssociation(store, { id: "malformed", lineageId: "lin-e", sha: maliciousSha });
|
||||
insertAssociation(store, { id: "already-populated", lineageId: "lin-f", sha: addOnlySha, additions: 99, deletions: 88 });
|
||||
const populatedBefore = readStats(store, "already-populated");
|
||||
|
||||
const dryRun = await store.backfillCommitAssociationDiffStats({ dryRun: true });
|
||||
expect(dryRun).toEqual({
|
||||
scannedRows: 5,
|
||||
distinctCommits: 4,
|
||||
updatedRows: 3,
|
||||
skippedUnavailableCommits: 1,
|
||||
skippedInvalidShas: 1,
|
||||
dryRun: true,
|
||||
});
|
||||
expect(readStats(store, "null-add-1")).toMatchObject({ additions: null, deletions: null });
|
||||
expect(readStats(store, "unavailable")).toMatchObject({ additions: null, deletions: null });
|
||||
|
||||
const report = await store.backfillCommitAssociationDiffStats({ dryRun: false });
|
||||
expect(report).toEqual({
|
||||
scannedRows: 5,
|
||||
distinctCommits: 4,
|
||||
updatedRows: 3,
|
||||
skippedUnavailableCommits: 1,
|
||||
skippedInvalidShas: 1,
|
||||
dryRun: false,
|
||||
});
|
||||
|
||||
expect(readStats(store, "null-add-1")).toMatchObject({ additions: 1, deletions: 0 });
|
||||
expect(readStats(store, "null-add-2")).toMatchObject({ additions: 1, deletions: 0 });
|
||||
expect(readStats(store, "null-delete")).toMatchObject({ additions: 0, deletions: 2 });
|
||||
expect(readStats(store, "unavailable")).toMatchObject({ additions: null, deletions: null });
|
||||
expect(readStats(store, "malformed")).toMatchObject({ additions: null, deletions: null });
|
||||
expect(readStats(store, "already-populated")).toEqual(populatedBefore);
|
||||
|
||||
const secondRun = await store.backfillCommitAssociationDiffStats({ dryRun: false });
|
||||
expect(secondRun).toEqual({
|
||||
scannedRows: 2,
|
||||
distinctCommits: 2,
|
||||
updatedRows: 0,
|
||||
skippedUnavailableCommits: 1,
|
||||
skippedInvalidShas: 1,
|
||||
dryRun: false,
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -47,6 +47,7 @@ export type {
|
||||
TaskCommitAssociation,
|
||||
TaskCommitAssociationConfidence,
|
||||
TaskCommitAssociationMatchSource,
|
||||
CommitAssociationDiffBackfillReport,
|
||||
PluginActivation,
|
||||
PluginActivationInput,
|
||||
} from "./types.js";
|
||||
|
||||
@@ -3,7 +3,7 @@ import { randomUUID } from "node:crypto";
|
||||
import { mkdir, readdir, readFile, stat, writeFile, rename, unlink } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { existsSync, watch, type Dirent, type FSWatcher } from "node:fs";
|
||||
import type { Task, TaskDetail, TaskCreateInput, TaskAttachment, AgentLogEntry, BoardConfig, Column, ColumnId, CheckoutClaimPrecondition, MergeResult, Settings, GlobalSettings, ProjectSettings, ActivityLogEntry, ActivityEventType, TaskDocument, TaskDocumentRevision, TaskDocumentCreateInput, TaskDocumentWithTask, Artifact, ArtifactCreateInput, ArtifactType, ArtifactWithTask, InboxTask, TaskLogEntry, RunMutationContext, RunAuditEvent, RunAuditEventInput, RunAuditEventFilter, ArchivedTaskEntry, ArchiveAgentLogMode, TaskPriority, SourceType, WorkflowStepTemplate, Agent, AutostashOrphanRecord, TaskCommitAssociation, TaskCommitAssociationMatchSource, TaskCommitAssociationConfidence, GithubIssueAction, MergeQueueEntry, MergeQueueEnqueueOptions, MergeQueueAcquireOptions, MergeQueueReleaseOutcome, HandoffToReviewOptions, GoalCitation, GoalCitationFilter, GoalCitationInput, GoalCitationSurface, BranchGroup, BranchGroupCreateInput, BranchGroupUpdate, TaskBranchAssignmentMode, MergeRequestRecord, MergeRequestState, MergeRequestWorkflowProjectionOptions, CompletionHandoffMarker, WorkflowWorkItem, WorkflowWorkItemDueFilter, WorkflowWorkItemKind, WorkflowWorkItemState, WorkflowWorkItemTransitionPatch, WorkflowWorkItemUpsertInput, PrEntity, PrEntityCreateInput, PrEntityUpdate, PrEntityState, PrThreadState, PrThreadOutcome, PrConflictState, PrChecksRollup, PrReviewDecision, PluginActivation, PluginActivationInput } from "./types.js";
|
||||
import type { Task, TaskDetail, TaskCreateInput, TaskAttachment, AgentLogEntry, BoardConfig, Column, ColumnId, CheckoutClaimPrecondition, MergeResult, Settings, GlobalSettings, ProjectSettings, ActivityLogEntry, ActivityEventType, TaskDocument, TaskDocumentRevision, TaskDocumentCreateInput, TaskDocumentWithTask, Artifact, ArtifactCreateInput, ArtifactType, ArtifactWithTask, InboxTask, TaskLogEntry, RunMutationContext, RunAuditEvent, RunAuditEventInput, RunAuditEventFilter, ArchivedTaskEntry, ArchiveAgentLogMode, TaskPriority, SourceType, WorkflowStepTemplate, Agent, AutostashOrphanRecord, TaskCommitAssociation, TaskCommitAssociationMatchSource, TaskCommitAssociationConfidence, CommitAssociationDiffBackfillReport, GithubIssueAction, MergeQueueEntry, MergeQueueEnqueueOptions, MergeQueueAcquireOptions, MergeQueueReleaseOutcome, HandoffToReviewOptions, GoalCitation, GoalCitationFilter, GoalCitationInput, GoalCitationSurface, BranchGroup, BranchGroupCreateInput, BranchGroupUpdate, TaskBranchAssignmentMode, MergeRequestRecord, MergeRequestState, MergeRequestWorkflowProjectionOptions, CompletionHandoffMarker, WorkflowWorkItem, WorkflowWorkItemDueFilter, WorkflowWorkItemKind, WorkflowWorkItemState, WorkflowWorkItemTransitionPatch, WorkflowWorkItemUpsertInput, PrEntity, PrEntityCreateInput, PrEntityUpdate, PrEntityState, PrThreadState, PrThreadOutcome, PrConflictState, PrChecksRollup, PrReviewDecision, PluginActivation, PluginActivationInput } from "./types.js";
|
||||
import { createActivityLogSnapshot, createRunAuditSnapshot, createTaskMetadataSnapshot, toTaskMetadataRecord, validateSnapshotEnvelope, type ActivityLogSnapshot, type RunAuditSnapshot, type TaskMetadataSnapshot } from "./shared-mesh-state.js";
|
||||
import { VALID_TRANSITIONS, COLUMNS, DEFAULT_SETTINGS, isColumn, isGlobalOnlySettingsKey, WORKFLOW_STEP_TEMPLATES, validateDocumentKey } from "./types.js";
|
||||
import { DEFAULT_PROJECT_SETTINGS } from "./settings-schema.js";
|
||||
@@ -569,6 +569,11 @@ interface TaskCommitAssociationRow {
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
interface CommitAssociationDiffBackfillCandidateRow {
|
||||
commitSha: string;
|
||||
rowCount: number;
|
||||
}
|
||||
|
||||
interface TaskDocumentRow {
|
||||
id: string;
|
||||
taskId: string;
|
||||
@@ -16847,6 +16852,75 @@ ${notificationsSection}`;
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* FNXC:CommandCenterLocBackfill 2026-06-19-12:30:
|
||||
* Historical LOC backfill is an explicit operator action that fills only rows where both diff-stat columns are NULL. FN-6704 writes additions/deletions atomically, so candidate selection and updates guard on both columns to stay idempotent and avoid overwriting already-captured stats. Stored SHAs are untrusted; validate them before git interpolation. Unavailable commit objects remain NULL because NULL means "stats unknown" while 0 is a real zero-line stat. Dry-run reports the rows that would be updated without writing them.
|
||||
*/
|
||||
async backfillCommitAssociationDiffStats(
|
||||
options: { dryRun?: boolean } = {},
|
||||
): Promise<CommitAssociationDiffBackfillReport> {
|
||||
const dryRun = options.dryRun === true;
|
||||
const candidates = this.db.prepare(
|
||||
`SELECT commitSha, COUNT(*) AS rowCount
|
||||
FROM task_commit_associations
|
||||
WHERE additions IS NULL AND deletions IS NULL
|
||||
GROUP BY commitSha
|
||||
ORDER BY commitSha`,
|
||||
).all() as CommitAssociationDiffBackfillCandidateRow[];
|
||||
|
||||
const report: CommitAssociationDiffBackfillReport = {
|
||||
scannedRows: candidates.reduce((sum, row) => sum + row.rowCount, 0),
|
||||
distinctCommits: candidates.length,
|
||||
updatedRows: 0,
|
||||
skippedUnavailableCommits: 0,
|
||||
skippedInvalidShas: 0,
|
||||
dryRun,
|
||||
};
|
||||
|
||||
const validShaPattern = /^[0-9a-fA-F]{7,64}$/;
|
||||
const updateStats = this.db.prepare(
|
||||
`UPDATE task_commit_associations
|
||||
SET additions = ?, deletions = ?, updatedAt = ?
|
||||
WHERE commitSha = ? AND additions IS NULL AND deletions IS NULL`,
|
||||
);
|
||||
|
||||
for (const candidate of candidates) {
|
||||
const commitSha = candidate.commitSha;
|
||||
if (!validShaPattern.test(commitSha)) {
|
||||
report.skippedInvalidShas += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
const verify = await this.runGitCommand(`git cat-file -e ${commitSha}^{commit}`);
|
||||
if (verify.exitCode !== 0) {
|
||||
report.skippedUnavailableCommits += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
const statsResult = await this.runGitCommand(`git show --shortstat --format= ${commitSha}`);
|
||||
if (statsResult.exitCode !== 0) {
|
||||
report.skippedUnavailableCommits += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
const normalized = statsResult.stdout.trim().replace(/\n/g, " ");
|
||||
const insertionsMatch = normalized.match(/(\d+) insertions?\(\+\)/);
|
||||
const deletionsMatch = normalized.match(/(\d+) deletions?\(-\)/);
|
||||
const additions = insertionsMatch ? Number.parseInt(insertionsMatch[1], 10) : 0;
|
||||
const deletions = deletionsMatch ? Number.parseInt(deletionsMatch[1], 10) : 0;
|
||||
|
||||
if (dryRun) {
|
||||
report.updatedRows += candidate.rowCount;
|
||||
continue;
|
||||
}
|
||||
|
||||
const result = updateStats.run(additions, deletions, new Date().toISOString(), commitSha);
|
||||
report.updatedRows += Number(result.changes);
|
||||
}
|
||||
|
||||
return report;
|
||||
}
|
||||
|
||||
async replaceLegacyTaskCommitAssociations(
|
||||
lineageId: string,
|
||||
associations: Array<Omit<TaskCommitAssociation, "id" | "createdAt" | "updatedAt" | "taskLineageId">>,
|
||||
|
||||
@@ -4542,6 +4542,15 @@ export interface TaskCommitAssociation {
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
export interface CommitAssociationDiffBackfillReport {
|
||||
scannedRows: number;
|
||||
distinctCommits: number;
|
||||
updatedRows: number;
|
||||
skippedUnavailableCommits: number;
|
||||
skippedInvalidShas: number;
|
||||
dryRun: boolean;
|
||||
}
|
||||
|
||||
export const COLUMN_LABELS: Record<Column, string> = {
|
||||
triage: "Planning",
|
||||
todo: "Todo",
|
||||
|
||||
@@ -92,6 +92,7 @@ import type {
|
||||
WorkflowSettingOption,
|
||||
WorkflowSettingRender,
|
||||
WorkflowSettingRejection,
|
||||
CommitAssociationDiffBackfillReport,
|
||||
} from "@fusion/core";
|
||||
import type { PlanningQuestion, PlanningSummary } from "@fusion/core";
|
||||
import type { GithubIssueAction, ScheduledTask, ScheduledTaskCreateInput, ScheduledTaskUpdateInput, AutomationRunResult, Routine, RoutineCreateInput, RoutineUpdateInput, RoutineExecutionResult } from "@fusion/core";
|
||||
@@ -113,6 +114,7 @@ export type FetchOptions = DedupeOptions;
|
||||
|
||||
// Re-export skills types for use by hooks and components
|
||||
export type { DiscoveredSkill, CatalogEntry, CatalogFetchResult, ToggleSkillResult, SkillContent, SkillFileEntry };
|
||||
export type { CommitAssociationDiffBackfillReport };
|
||||
|
||||
export class ApiRequestError extends Error {
|
||||
readonly status: number;
|
||||
@@ -7673,6 +7675,20 @@ export function backfillMissionAssertions(
|
||||
);
|
||||
}
|
||||
|
||||
/** Backfill historical Command Center LOC stats for commit associations. Defaults to dry-run. */
|
||||
export function backfillCommitAssociationDiffStats(
|
||||
options?: { dryRun?: boolean },
|
||||
projectId?: string,
|
||||
): Promise<CommitAssociationDiffBackfillReport> {
|
||||
return api<CommitAssociationDiffBackfillReport>(
|
||||
withProjectId("/command-center/productivity/backfill-loc", projectId),
|
||||
{
|
||||
method: "POST",
|
||||
body: JSON.stringify({ dryRun: options?.dryRun ?? true }),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
/** Query options for paginated mission event logs. */
|
||||
export interface MissionEventQueryOptions {
|
||||
limit?: number;
|
||||
|
||||
@@ -61,8 +61,6 @@ export function ProductivityArea({ range }: { range: DateRange }) {
|
||||
[data?.byLanguage],
|
||||
);
|
||||
|
||||
const loc = data?.loc ?? { value: null, unavailable: true };
|
||||
const hoursSaved = data?.hoursSaved ?? { value: null, unavailable: true };
|
||||
const taskDuration = data?.taskDuration ?? {
|
||||
completedTasks: 0,
|
||||
averageMs: null,
|
||||
@@ -71,6 +69,12 @@ export function ProductivityArea({ range }: { range: DateRange }) {
|
||||
totalMs: null,
|
||||
unavailable: true,
|
||||
};
|
||||
/*
|
||||
FNXC:CommandCenterProductivity 2026-06-22-00:32:
|
||||
Backfill-era dashboard tests and cached clients can render ProductivityArea with legacy productivity payloads that predate LOC and hours-saved summaries. Treat missing nested summaries as unavailable sentinels so the whole Command Center remains mounted instead of crashing during responsive-layout verification.
|
||||
*/
|
||||
const loc = data?.loc ?? { value: null, unavailable: true };
|
||||
const hoursSaved = data?.hoursSaved ?? { value: null, unavailable: true };
|
||||
const isEmpty =
|
||||
!data ||
|
||||
(data.modifiedFiles === 0 &&
|
||||
|
||||
@@ -56,19 +56,31 @@ class MockStore extends EventEmitter {
|
||||
async listTasks(): Promise<Task[]> {
|
||||
return [];
|
||||
}
|
||||
|
||||
async backfillCommitAssociationDiffStats() {
|
||||
return {
|
||||
scannedRows: 0,
|
||||
distinctCommits: 0,
|
||||
updatedRows: 0,
|
||||
skippedUnavailableCommits: 0,
|
||||
skippedInvalidShas: 0,
|
||||
dryRun: true,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const TOKEN = "fn_cc_test1234567890abcdef";
|
||||
const ENDPOINTS = [
|
||||
"/api/command-center/tokens",
|
||||
"/api/command-center/tools",
|
||||
"/api/command-center/activity",
|
||||
"/api/command-center/productivity",
|
||||
"/api/command-center/plugin-activations",
|
||||
"/api/command-center/team",
|
||||
"/api/command-center/github",
|
||||
"/api/command-center/signals",
|
||||
"/api/command-center/live",
|
||||
const ENDPOINTS: Array<{ method?: "GET" | "POST"; path: string }> = [
|
||||
{ path: "/api/command-center/tokens" },
|
||||
{ path: "/api/command-center/tools" },
|
||||
{ path: "/api/command-center/activity" },
|
||||
{ path: "/api/command-center/productivity" },
|
||||
{ method: "POST", path: "/api/command-center/productivity/backfill-loc" },
|
||||
{ path: "/api/command-center/plugin-activations" },
|
||||
{ path: "/api/command-center/team" },
|
||||
{ path: "/api/command-center/github" },
|
||||
{ path: "/api/command-center/signals" },
|
||||
{ path: "/api/command-center/live" },
|
||||
];
|
||||
|
||||
describe("Command Center routes — auth", () => {
|
||||
@@ -80,9 +92,10 @@ describe("Command Center routes — auth", () => {
|
||||
const app = createServer(new MockStore() as unknown as TaskStore, {
|
||||
daemon: { token: TOKEN },
|
||||
});
|
||||
for (const path of ENDPOINTS) {
|
||||
const res = await request(app, "GET", path);
|
||||
expect(res.status, `${path} should be 401 unauthenticated`).toBe(401);
|
||||
for (const endpoint of ENDPOINTS) {
|
||||
const method = endpoint.method ?? "GET";
|
||||
const res = await request(app, method, endpoint.path);
|
||||
expect(res.status, `${method} ${endpoint.path} should be 401 unauthenticated`).toBe(401);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -90,11 +103,14 @@ describe("Command Center routes — auth", () => {
|
||||
const app = createServer(new MockStore() as unknown as TaskStore, {
|
||||
daemon: { token: TOKEN },
|
||||
});
|
||||
for (const path of ENDPOINTS) {
|
||||
const res = await request(app, "GET", path, undefined, {
|
||||
for (const endpoint of ENDPOINTS) {
|
||||
const method = endpoint.method ?? "GET";
|
||||
const body = method === "POST" ? JSON.stringify({}) : undefined;
|
||||
const res = await request(app, method, endpoint.path, body, {
|
||||
Authorization: `Bearer ${TOKEN}`,
|
||||
...(method === "POST" ? { "content-type": "application/json" } : {}),
|
||||
});
|
||||
expect(res.status, `${path} should be 200 with token`).toBe(200);
|
||||
expect(res.status, `${method} ${endpoint.path} should be 200 with token`).toBe(200);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
// @vitest-environment node
|
||||
|
||||
import express, { type NextFunction, type Request, type Response } from "express";
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { rmSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
@@ -196,10 +196,11 @@ function buildApp(stores: Record<string, TaskStore>, fallback: TaskStore) {
|
||||
return app;
|
||||
}
|
||||
|
||||
/** A minimal TaskStore exposing only getDatabase(), which is all the routes use. */
|
||||
function storeFor(db: Database): TaskStore {
|
||||
/** A minimal TaskStore exposing only the methods Command Center routes use. */
|
||||
function storeFor(db: Database, overrides: Partial<TaskStore> = {}): TaskStore {
|
||||
const store = new EventEmitter() as unknown as TaskStore & { getDatabase(): Database };
|
||||
store.getDatabase = () => db;
|
||||
Object.assign(store, overrides);
|
||||
return store;
|
||||
}
|
||||
|
||||
@@ -357,6 +358,48 @@ describe("register-command-center-routes", () => {
|
||||
expect(signals.body).toHaveProperty("bySeverity");
|
||||
});
|
||||
|
||||
it("runs the productivity LOC backfill route as a dry-run by default and respects writes", async () => {
|
||||
const backfill = vi.fn(async (options?: { dryRun?: boolean }) => ({
|
||||
scannedRows: 3,
|
||||
distinctCommits: 2,
|
||||
updatedRows: options?.dryRun === false ? 3 : 0,
|
||||
skippedUnavailableCommits: 1,
|
||||
skippedInvalidShas: 0,
|
||||
dryRun: options?.dryRun ?? true,
|
||||
}));
|
||||
const scopedStore = storeFor(dbA, { backfillCommitAssociationDiffStats: backfill } as unknown as Partial<TaskStore>);
|
||||
const scopedApp = buildApp({ "proj-a": scopedStore }, scopedStore);
|
||||
|
||||
const preview = await request(
|
||||
scopedApp,
|
||||
"POST",
|
||||
"/api/command-center/productivity/backfill-loc?projectId=proj-a",
|
||||
JSON.stringify({}),
|
||||
{ "content-type": "application/json" },
|
||||
);
|
||||
expect(preview.status).toBe(200);
|
||||
expect(preview.body).toMatchObject({
|
||||
scannedRows: 3,
|
||||
distinctCommits: 2,
|
||||
updatedRows: 0,
|
||||
skippedUnavailableCommits: 1,
|
||||
skippedInvalidShas: 0,
|
||||
dryRun: true,
|
||||
});
|
||||
expect(backfill).toHaveBeenLastCalledWith({ dryRun: true });
|
||||
|
||||
const write = await request(
|
||||
scopedApp,
|
||||
"POST",
|
||||
"/api/command-center/productivity/backfill-loc?projectId=proj-a",
|
||||
JSON.stringify({ dryRun: false }),
|
||||
{ "content-type": "application/json" },
|
||||
);
|
||||
expect(write.status).toBe(200);
|
||||
expect(write.body).toMatchObject({ updatedRows: 3, dryRun: false });
|
||||
expect(backfill).toHaveBeenLastCalledWith({ dryRun: false });
|
||||
});
|
||||
|
||||
it("returns the live snapshot shape", async () => {
|
||||
const res = await request(app, "GET", "/api/command-center/live?projectId=proj-a");
|
||||
expect(res.status).toBe(200);
|
||||
|
||||
@@ -243,6 +243,26 @@ export const registerCommandCenterRoutes: ApiRouteRegistrar = (ctx) => {
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* POST /api/command-center/productivity/backfill-loc
|
||||
* Explicit operator action to backfill historical commit-association LOC stats.
|
||||
*
|
||||
* FNXC:CommandCenterLocBackfill 2026-06-21-00:00:
|
||||
* The LOC backfill must never run during render-time analytics reads. Keep it an authenticated operator POST, resolve the project-scoped store before invoking the git-backed store method, and default to dry-run so operators can preview historical NULL-only updates before writing.
|
||||
*/
|
||||
router.post("/command-center/productivity/backfill-loc", async (req, res) => {
|
||||
try {
|
||||
const store = await getScopedStore(req);
|
||||
const body = (req.body ?? {}) as { dryRun?: unknown };
|
||||
const dryRun = typeof body.dryRun === "boolean" ? body.dryRun : true;
|
||||
const result = await store.backfillCommitAssociationDiffStats({ dryRun });
|
||||
res.json(result);
|
||||
} catch (err: unknown) {
|
||||
if (err instanceof ApiError) throw err;
|
||||
rethrowAsApiError(err, "Failed to backfill productivity LOC stats");
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* GET /api/command-center/team
|
||||
* Per-agent store-derived tokens/cost, files changed, task counts, and live identity.
|
||||
|
||||
Reference in New Issue
Block a user