diff --git a/.changeset/fn-6675-github-closed-at-backfill-dashboard.md b/.changeset/fn-6675-github-closed-at-backfill-dashboard.md new file mode 100644 index 0000000000..e54ebbbcea --- /dev/null +++ b/.changeset/fn-6675-github-closed-at-backfill-dashboard.md @@ -0,0 +1,5 @@ +--- +"@runfusion/fusion": minor +--- + +Add a Command Center GitHub affordance for operators to run the historical source-issue closed-at backfill and review accumulated scanned, filled, skipped, and error counts. diff --git a/docs/dashboard-guide.md b/docs/dashboard-guide.md index f9b752b759..930a7f7fe8 100644 --- a/docs/dashboard-guide.md +++ b/docs/dashboard-guide.md @@ -670,7 +670,7 @@ Features: - **Productivity** separates outcome counters (commits and pull requests) from volume proxies such as modified files, lines changed, and files by language. - **Team** shows a per-agent analytics table plus tokens-by-agent and tasks-done-by-agent charts. Metrics come only from the project-scoped `tasks` and `agents` tables: token totals and estimated cost are summed from the `tokenUsage*` columns by `assignedAgentId`, files changed counts parsed `tasks.modifiedFiles` paths, tasks done counts `column = 'done'` moves in the selected range, and in-progress / in-review values reflect current task columns. Agent name, role, and live state come from the `agents` table; deleted-agent task history falls back to the raw agent id instead of crashing. The tab uses `/api/command-center/team`, adds no schema, never calls GitHub, and intentionally leaves per-agent issues filed/fixed to FN-6653. Decorative chart reveal motion uses duration tokens and is disabled for reduced-motion users. - **Ecosystem** shows active model breadth and per-model task activity; unavailable plugin-activation metrics render as unavailable rather than zero. -- **GitHub** shows local GitHub issue flow for the selected range: **Filed by Fusion** counts tasks with a persisted `githubTracking.issue`, **Fixed by Fusion** counts tasks imported from GitHub source issues (`sourceIssueProvider = "github"`) that are currently in `done`, using the persisted `sourceIssueClosedAt` / `TaskSourceIssue.closedAt` close time when the reconciler has observed it. Rows that predate the field or have not been observed closed fall back to task `updatedAt` as the documented completion-time approximation; Fusion never fabricates a close timestamp and this analytics path never calls GitHub, the `gh` CLI, or any external network source. To make historical fixed dates exact, an operator can run the project-scoped manual `POST /api/git/github/backfill-source-issue-closed-at` endpoint with optional `{ "offset": 0, "limit": 200 }` batches until `hasMore` is false; the endpoint fetches real GitHub `closed_at` values once, fills only missing `sourceIssueClosedAt` values, and never runs automatically. The area shows filed/fixed/net stat cards, filed-vs-fixed daily sparklines, and a by-repository bar breakdown. +- **GitHub** shows local GitHub issue flow for the selected range: **Filed by Fusion** counts tasks with a persisted `githubTracking.issue`, **Fixed by Fusion** counts tasks imported from GitHub source issues (`sourceIssueProvider = "github"`) that are currently in `done`, using the persisted `sourceIssueClosedAt` / `TaskSourceIssue.closedAt` close time when the reconciler has observed it. Rows that predate the field or have not been observed closed fall back to task `updatedAt` as the documented completion-time approximation; Fusion never fabricates a close timestamp and this analytics path never calls GitHub, the `gh` CLI, or any external network source. To make historical fixed dates exact, use **Backfill exact close times** in the Fixed by Fusion card; the dashboard calls the project-scoped manual `POST /api/git/github/backfill-source-issue-closed-at` endpoint in `{ offset, limit }` batches until `hasMore` is false, then surfaces the accumulated `scanned`, `filled`, `skipped`, and `errors` counts. The endpoint fetches real GitHub `closed_at` values once, fills only missing `sourceIssueClosedAt` values, and never runs automatically or from analytics-time rendering. The area shows filed/fixed/net stat cards, filed-vs-fixed daily sparklines, and a by-repository bar breakdown. - **Signals** shows external signal totals, open/resolved counts, MTTR, and source/severity breakdowns when signal sources are connected. - **System** is the canonical system-telemetry destination. It reuses `GET /api/system-stats` with no new endpoint, renders live radial gauges for app CPU, host memory, and heap usage, keeps a small client-side rolling buffer for CPU/memory trend sparklines, and charts tasks by column plus agents by state with the shared Command Center chart primitives. The Vitest process count, manual kill confirmation, auto-kill toggle, threshold controls, and last-auto-kill timestamp moved here unchanged; the standalone System Stats modal and its desktop Header/mobile More affordances were removed. - **Mission Control** shows live active sessions/runs/nodes, current sessions and nodes, an animated live activity snapshot, and a live SDLC funnel; when idle it reports that live updates resume when work starts. Motion-heavy accents respect reduced-motion preferences. @@ -682,7 +682,7 @@ Rendering invariants: Data states: - Overview shows a loading state while core analytics settle, then shows `No usage data yet. Run some agents to populate the Command Center.` only after the selected range has settled with no core usage data. -- GitHub issue analytics is local and additive: empty filed/fixed totals render the GitHub area's empty state; malformed historical `githubTracking` JSON is skipped instead of breaking the Command Center. +- GitHub issue analytics is local and additive: empty filed/fixed totals keep the stat cards and historical backfill button available while omitting empty chart shells; malformed historical `githubTracking` JSON is skipped instead of breaking the Command Center. - Team analytics renders its shared loading/error/empty states for null or zero-agent responses, omits empty chart shells for zero-value datasets, and keeps the Command Center tab panel as the mobile scroll owner. - System telemetry keeps the previous snapshot visible during refresh failures, renders a first-sample CPU `Sampling…` state without NaN values, shows zero-value task/agent bars for empty collections, and keeps the Command Center tab panel as the mobile scroll owner. - Signals is best-effort: if the Signals endpoint is absent or no signal source is connected, the Signals area falls back to its empty state and other Command Center metrics remain valid. diff --git a/docs/storage.md b/docs/storage.md index 58e25495b7..6745137cb2 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -388,7 +388,7 @@ FN-5240/FN-5241/FN-5242 establish the handoff invariant: the only legal executor The `tasks.githubTracking` JSON column stores per-task GitHub tracking state (`enabled`, optional `repoOverride`, linked issue metadata, and `unlinkedAt`). It is additive and default-off; imported-source issue metadata remains in `issueInfo` / `sourceIssue`. Behavior wiring (issue creation/lifecycle sync and UI surfacing) lands in FN-3870/FN-3873/FN-3874. -The `tasks.sourceIssueClosedAt` column (migration 122) backs `TaskSourceIssue.closedAt`, a nullable ISO-8601 timestamp for the originating external issue's real close time. Going forward, the GitHub source-issue reconciler fills it when it closes the linked issue itself or observes GitHub's `closed_at`/`closedAt` value. Historical GitHub-imported `done`/`archived` rows that still have `NULL` can be filled retroactively by the optional manual `POST /api/git/github/backfill-source-issue-closed-at` sweep; the sweep is idempotent, paginated, writes only real GitHub `closed_at` values, and never overwrites an existing timestamp. Command Center "Fixed by Fusion" analytics read this exact timestamp when available and fall back to `updatedAt` only when it has not been observed. +The `tasks.sourceIssueClosedAt` column (migration 122) backs `TaskSourceIssue.closedAt`, a nullable ISO-8601 timestamp for the originating external issue's real close time. Going forward, the GitHub source-issue reconciler fills it when it closes the linked issue itself or observes GitHub's `closed_at`/`closedAt` value. Historical GitHub-imported `done`/`archived` rows that still have `NULL` can be filled retroactively by the optional manual `POST /api/git/github/backfill-source-issue-closed-at` sweep, now exposed as **Backfill exact close times** in the Command Center GitHub area's Fixed by Fusion card. The sweep is idempotent, paginated, writes only real GitHub `closed_at` values, reports `scanned`/`filled`/`skipped`/`errors`, and never overwrites an existing timestamp or runs automatically. Command Center "Fixed by Fusion" analytics read this exact timestamp when available and fall back to `updatedAt` only when it has not been observed. The `tasks.tokenUsage*` columns store cumulative per-task token usage for analytics. `tokenUsageModelProvider` and `tokenUsageModelId` are analytics-only snapshots of the actually-used runtime model recorded when usage is accumulated; they let Command Center group and price resolved-via-settings usage by provider/model without writing the task-level `modelProvider` / `modelId` own-model override fields that control future model resolution. Cost attribution reads the snapshot first and falls back to the legacy own-model columns for pre-snapshot rows. | `config` | Single-row project configuration (`nextId`, settings payload, workflow step counters). | diff --git a/packages/dashboard/app/api/legacy.ts b/packages/dashboard/app/api/legacy.ts index a7a22f6d80..489c95c7e5 100644 --- a/packages/dashboard/app/api/legacy.ts +++ b/packages/dashboard/app/api/legacy.ts @@ -7196,6 +7196,14 @@ export interface KillVitestResponse { pids: number[]; } +export interface GithubSourceIssueClosedAtBackfillResult { + scanned: number; + filled: number; + skipped: number; + errors: number; + hasMore: boolean; +} + export function fetchSystemStats(projectId?: string): Promise { return api(withProjectId("/system-stats", projectId)); } @@ -7206,6 +7214,23 @@ export function killVitestProcesses(projectId?: string): Promise { + return api( + withProjectId("/git/github/backfill-source-issue-closed-at", projectId), + { + method: "POST", + body: JSON.stringify({ offset: options.offset, limit: options.limit }), + }, + ); +} + /** Fetch unified activity feed */ export function fetchActivityFeed(options?: FeedOptions): Promise { const params = new URLSearchParams(); diff --git a/packages/dashboard/app/components/command-center/CommandCenter.css b/packages/dashboard/app/components/command-center/CommandCenter.css index d26030c737..21be48182e 100644 --- a/packages/dashboard/app/components/command-center/CommandCenter.css +++ b/packages/dashboard/app/components/command-center/CommandCenter.css @@ -129,6 +129,45 @@ Command Center chart/stat surfaces share one tokenized card rhythm so new chart text-align: center; } +/* +FNXC:CommandCenterGithub 2026-06-18-19:27: +The GitHub closed-at backfill control lives inside the existing Fixed by Fusion card and must use tokenized spacing/status colors so the operator result row stays readable on desktop and mobile without creating a separate layout surface. +*/ +.cc-github-backfill-actions { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: var(--space-2); +} + +.cc-github-backfill-status { + display: flex; + flex-direction: column; + gap: var(--space-1); + color: var(--text-muted); + font-size: var(--font-size-xs); +} + +.cc-github-backfill-status--error { + color: var(--color-error); +} + +.cc-github-backfill-status--warning { + color: var(--color-warning); +} + +@media (max-width: 768px) { + .cc-github-backfill-actions { + align-items: stretch; + flex-direction: column; + } + + .cc-github-backfill-actions .btn { + justify-content: center; + inline-size: 100%; + } +} + /* FNXC:CommandCenterStyling 2026-06-18-00:00: Command Center live/chart containers must shrink inside the mobile tabpanel without creating a second scroll owner or clipping chart height (FN-6664). diff --git a/packages/dashboard/app/components/command-center/areas/GithubArea.tsx b/packages/dashboard/app/components/command-center/areas/GithubArea.tsx index b18c78c40b..871b51d510 100644 --- a/packages/dashboard/app/components/command-center/areas/GithubArea.tsx +++ b/packages/dashboard/app/components/command-center/areas/GithubArea.tsx @@ -2,9 +2,12 @@ FNXC:CommandCenterGithub 2026-06-18-00:00: The GitHub Command Center area visualizes only locally persisted task-store data: filed issues come from `githubTracking.issue`, and fixed issues are source-GitHub tasks currently in `done` using `updatedAt` as the documented completion approximation. No GitHub API or `gh` CLI calls belong in this rendering path. */ -import { useMemo } from "react"; +import { useCallback, useMemo, useState } from "react"; import { useTranslation } from "react-i18next"; +import { RefreshCw } from "lucide-react"; import type { GithubIssueAnalytics } from "@fusion/core"; +import { apiBackfillGithubSourceIssueClosedAt } from "../../../api/legacy"; +import type { GithubSourceIssueClosedAtBackfillResult } from "../../../api/legacy"; import type { DateRange } from "../DateRangePicker"; import { Bar } from "../charts/Bar"; import { Sparkline } from "../charts/Sparkline"; @@ -12,12 +15,67 @@ import { AreaShell } from "./AreaShell"; import { useAnalyticsArea } from "./useAnalyticsArea"; import { formatCount } from "./areaShared"; +const GITHUB_SOURCE_ISSUE_BACKFILL_LIMIT = 100; +const GITHUB_SOURCE_ISSUE_BACKFILL_MAX_BATCHES = 1000; + +type BackfillAggregate = Omit; + export function GithubArea({ range }: { range: DateRange }) { const { t } = useTranslation("app"); const { data, isLoading, error } = useAnalyticsArea( "/command-center/github", range, ); + const [isBackfilling, setIsBackfilling] = useState(false); + const [backfillResult, setBackfillResult] = useState(null); + const [backfillError, setBackfillError] = useState(null); + + const handleBackfill = useCallback(async () => { + if (isBackfilling) return; + + setIsBackfilling(true); + setBackfillResult(null); + setBackfillError(null); + + try { + let offset = 0; + const aggregate: BackfillAggregate = { scanned: 0, filled: 0, skipped: 0, errors: 0 }; + + for (let batch = 0; batch < GITHUB_SOURCE_ISSUE_BACKFILL_MAX_BATCHES; batch += 1) { + const result = await apiBackfillGithubSourceIssueClosedAt({ + offset, + limit: GITHUB_SOURCE_ISSUE_BACKFILL_LIMIT, + }); + aggregate.scanned += result.scanned; + aggregate.filled += result.filled; + aggregate.skipped += result.skipped; + aggregate.errors += result.errors; + + if (!result.hasMore) { + setBackfillResult(aggregate); + return; + } + + offset += GITHUB_SOURCE_ISSUE_BACKFILL_LIMIT; + } + + setBackfillResult(aggregate); + setBackfillError( + t( + "commandCenter.github.backfillMaxBatches", + "Backfill stopped after the safety limit; rerun after checking server logs.", + ), + ); + } catch (err) { + setBackfillError( + err instanceof Error + ? err.message + : t("commandCenter.github.backfillFailed", "Failed to backfill GitHub source issue close times"), + ); + } finally { + setIsBackfilling(false); + } + }, [isBackfilling, t]); const daily = useMemo(() => data?.daily ?? [], [data?.daily]); const byRepo = useMemo(() => data?.byRepo ?? [], [data?.byRepo]); @@ -46,17 +104,25 @@ export function GithubArea({ range }: { range: DateRange }) { const isEmpty = !data || (filed === 0 && fixed === 0); const hasDailyTrend = daily.length > 0; const hasRepoBreakdown = repoBars.length > 0; + const backfillStatusClass = backfillError || (backfillResult?.errors ?? 0) > 0 + ? "cc-github-backfill-status--error" + : isBackfilling + ? "cc-github-backfill-status--warning" + : ""; return (

{t("commandCenter.github.totalsTitle", "GitHub issue flow")}

+ {isEmpty ? ( + {t("commandCenter.github.empty", "No GitHub issue activity in the selected range.")} + ) : null}
{t("commandCenter.github.filed", "Filed by Fusion")}
@@ -68,6 +134,53 @@ export function GithubArea({ range }: { range: DateRange }) { {t("commandCenter.github.fixedApproximation", "Uses done tasks updated in range")} + {/* + FNXC:CommandCenterGithub 2026-06-18-19:24: + This backfill is an explicit operator action that paginates the FN-6674 endpoint until `hasMore === false`, then surfaces scanned/filled/skipped/errors. Keep GitHub network fetches here in the click handler only; Command Center analytics and render-time data loading must stay backed by local task-store data. + */} +
+ +
+ {isBackfilling || backfillResult || backfillError ? ( +
+ {isBackfilling ? ( + {t("commandCenter.github.backfillPending", "Backfill is running in paginated batches.")} + ) : null} + {backfillError ? {backfillError} : null} + {backfillResult ? ( + + {backfillResult.scanned === 0 && backfillResult.filled === 0 + ? t( + "commandCenter.github.backfillNothing", + "Nothing to backfill. Scanned {{scanned}}, filled {{filled}}, skipped {{skipped}}, errors {{errors}}.", + backfillResult, + ) + : t( + "commandCenter.github.backfillResult", + "Backfill complete. Scanned {{scanned}}, filled {{filled}}, skipped {{skipped}}, errors {{errors}}.", + backfillResult, + )} + + ) : null} +
+ ) : null}
{t("commandCenter.github.net", "Net")}
diff --git a/packages/dashboard/app/components/command-center/areas/__tests__/areas.test.tsx b/packages/dashboard/app/components/command-center/areas/__tests__/areas.test.tsx index b8b803f36d..82eb86ad26 100644 --- a/packages/dashboard/app/components/command-center/areas/__tests__/areas.test.tsx +++ b/packages/dashboard/app/components/command-center/areas/__tests__/areas.test.tsx @@ -7,8 +7,11 @@ import { render, screen, fireEvent, waitFor, within, act, renderHook } from "@te // Mock the api() helper so the areas fetch deterministic fixtures. const apiMock = vi.fn(); +const backfillGithubSourceIssueClosedAtMock = vi.fn(); vi.mock("../../../../api/legacy", () => ({ api: (path: string, opts?: RequestInit) => apiMock(path, opts), + apiBackfillGithubSourceIssueClosedAt: (options?: { offset?: number; limit?: number }, projectId?: string) => + backfillGithubSourceIssueClosedAtMock(options, projectId), })); import { TokensArea } from "../TokensArea"; @@ -139,6 +142,7 @@ function activityFixture() { beforeEach(() => { apiMock.mockReset(); + backfillGithubSourceIssueClosedAtMock.mockReset(); }); afterEach(() => { @@ -578,7 +582,9 @@ describe("GithubArea", () => { apiMock.mockResolvedValue({ ...githubFixture(), filed: 0, fixed: 0, net: 0, daily: [], byRepo: [] }); render(); - await screen.findByTestId("cc-area-github-empty"); + await screen.findByTestId("cc-area-github"); + expect(screen.getByTestId("cc-area-github").textContent).toContain("No GitHub issue activity"); + expect(screen.getByTestId("cc-github-backfill-button")).toBeTruthy(); expect(screen.queryByTestId("cc-github-daily-trend")).toBeNull(); expect(screen.queryByTestId("cc-github-by-repo")).toBeNull(); }); @@ -609,6 +615,138 @@ describe("GithubArea", () => { render(); await waitFor(() => expect(apiMock).not.toHaveBeenCalled()); }); + + it("runs a single backfill batch and renders accumulated result counts", async () => { + apiMock.mockResolvedValue(githubFixture()); + backfillGithubSourceIssueClosedAtMock.mockResolvedValueOnce({ + scanned: 4, + filled: 2, + skipped: 1, + errors: 0, + hasMore: false, + }); + + render(); + await screen.findByTestId("cc-area-github"); + + fireEvent.click(screen.getByTestId("cc-github-backfill-button")); + + await screen.findByText(/Backfill complete/i); + expect(backfillGithubSourceIssueClosedAtMock).toHaveBeenCalledWith({ offset: 0, limit: 100 }, undefined); + const result = screen.getByTestId("cc-github-backfill-result"); + expect(result.textContent).toContain("Scanned 4, filled 2, skipped 1, errors 0"); + }); + + it("paginates multi-batch backfills with advancing offsets and summed counts", async () => { + apiMock.mockResolvedValue(githubFixture()); + backfillGithubSourceIssueClosedAtMock + .mockResolvedValueOnce({ scanned: 100, filled: 4, skipped: 90, errors: 1, hasMore: true }) + .mockResolvedValueOnce({ scanned: 25, filled: 3, skipped: 22, errors: 0, hasMore: false }); + + render(); + await screen.findByTestId("cc-area-github"); + fireEvent.click(screen.getByTestId("cc-github-backfill-button")); + + await waitFor(() => expect(backfillGithubSourceIssueClosedAtMock).toHaveBeenCalledTimes(2)); + expect(backfillGithubSourceIssueClosedAtMock).toHaveBeenNthCalledWith(1, { offset: 0, limit: 100 }, undefined); + expect(backfillGithubSourceIssueClosedAtMock).toHaveBeenNthCalledWith(2, { offset: 100, limit: 100 }, undefined); + const result = await screen.findByTestId("cc-github-backfill-result"); + expect(result.textContent).toContain("Scanned 125, filled 7, skipped 112, errors 1"); + expect(result.className).toContain("cc-github-backfill-status--error"); + }); + + it("shows the all-zero backfill as nothing to backfill instead of an error", async () => { + apiMock.mockResolvedValue(githubFixture()); + backfillGithubSourceIssueClosedAtMock.mockResolvedValueOnce({ + scanned: 0, + filled: 0, + skipped: 0, + errors: 0, + hasMore: false, + }); + + render(); + await screen.findByTestId("cc-area-github"); + fireEvent.click(screen.getByTestId("cc-github-backfill-button")); + + const result = await screen.findByTestId("cc-github-backfill-result"); + expect(result.textContent).toContain("Nothing to backfill"); + expect(result.className).not.toContain("cc-github-backfill-status--error"); + }); + + it("surfaces nonzero backfill error counts without throwing", async () => { + apiMock.mockResolvedValue(githubFixture()); + backfillGithubSourceIssueClosedAtMock.mockResolvedValueOnce({ + scanned: 8, + filled: 1, + skipped: 5, + errors: 2, + hasMore: false, + }); + + render(); + await screen.findByTestId("cc-area-github"); + fireEvent.click(screen.getByTestId("cc-github-backfill-button")); + + const result = await screen.findByTestId("cc-github-backfill-result"); + expect(result.textContent).toContain("errors 2"); + expect(result.className).toContain("cc-github-backfill-status--error"); + }); + + it("captures endpoint failures in local error UI", async () => { + apiMock.mockResolvedValue(githubFixture()); + backfillGithubSourceIssueClosedAtMock.mockRejectedValueOnce(new Error("endpoint failed")); + + render(); + await screen.findByTestId("cc-area-github"); + fireEvent.click(screen.getByTestId("cc-github-backfill-button")); + + const result = await screen.findByTestId("cc-github-backfill-result"); + expect(result.textContent).toContain("endpoint failed"); + expect(result.className).toContain("cc-github-backfill-status--error"); + }); + + it("disables and guards the button while a backfill is in flight", async () => { + apiMock.mockResolvedValue(githubFixture()); + let resolveBackfill: ((value: { scanned: number; filled: number; skipped: number; errors: number; hasMore: boolean }) => void) | null = null; + backfillGithubSourceIssueClosedAtMock.mockImplementationOnce( + () => new Promise((resolve) => { + resolveBackfill = resolve; + }), + ); + + render(); + await screen.findByTestId("cc-area-github"); + const button = screen.getByTestId("cc-github-backfill-button") as HTMLButtonElement; + fireEvent.click(button); + + await waitFor(() => expect(button.disabled).toBe(true)); + fireEvent.click(button); + expect(backfillGithubSourceIssueClosedAtMock).toHaveBeenCalledTimes(1); + + resolveBackfill?.({ scanned: 1, filled: 1, skipped: 0, errors: 0, hasMore: false }); + await screen.findByText(/Backfill complete/i); + }); + + it("stops a pathological always-has-more response at the max iteration guard", async () => { + apiMock.mockResolvedValue(githubFixture()); + backfillGithubSourceIssueClosedAtMock.mockResolvedValue({ + scanned: 1, + filled: 0, + skipped: 1, + errors: 0, + hasMore: true, + }); + + render(); + await screen.findByTestId("cc-area-github"); + fireEvent.click(screen.getByTestId("cc-github-backfill-button")); + + const result = await screen.findByText(/safety limit/i); + expect(result.textContent).toContain("safety limit"); + expect(backfillGithubSourceIssueClosedAtMock).toHaveBeenCalledTimes(1000); + expect(screen.getByTestId("cc-github-backfill-result").textContent).toContain("Scanned 1000"); + }); }); describe("SignalsArea", () => {