# Migrate storage from SQLite to PostgreSQL — full dashboard cutover Migrates Fusion's storage layer to the embedded PostgreSQL `AsyncDataLayer` (the default backend) and **completes the satellite-store + feature cutover** so every dashboard and Command Center surface works in PG mode. ## Status — every surface works in embedded-PG mode Verified live against a running embedded-Postgres dashboard (all **200**, zero 5xx) and gate-tested (**23 files / 99 tests** on embedded PG, plus engine-core 294 and ci-shape 63 in the blocking merge gate; core/engine/cli/dashboard typecheck clean). | Area | Surfaces | State | |---|---|---| | Satellite stores | workflows, todos, insights, research, missions, goals, mailbox | ✅ | | Views | artifacts, documents, evals | ✅ | | Command Center | activity, productivity, team, tokens, tools, **workflows**, **github**, **signals**, **plugin-activations**, **live** (all 10) | ✅ | | Run execution | insight generation, research run execution | ✅ (store-path; AI step needs a provider) | | Live updates | SSE push for mission/research/insight events | ✅ | | Workflow editing | create / update / delete / select (+ id counter) | ✅ | | Engine | mission autopilot, incident-signal ingestion, regression storm-guard, agent wake-on-message | ✅ | | Core | tasks, agents, secrets, automations, memory, chat, usage, PRs, git | ✅ | ## Approach Each satellite store gets an `Async<Store>` wrapper exposing the sync store's method names over the existing `async-*-store.ts` helpers; `get<Store>Store()` returns a `Sync | Async` union; consumers `await` (harmless on sync), and engine/CLI paths that can't convert use `instanceof Sync` graceful fallback. Analytics aggregators branch on `"ping" in dbOrLayer` to run schema-qualified raw SQL over `project.*` (snake_case) in PG. Executors/orchestrators/autopilot are await-converted to drive the union store; the async store wrappers extend `EventEmitter` so SSE live-push fires in both backends. Not-yet-ported capabilities degrade gracefully (never 500) and are individually called out in commits. ## Sync with main The branch is kept continuously merged with `main` (currently through FN-7845, 2026-07-12); the earlier "final rebase deferred" note no longer applies. Use **Create a merge commit** (or squash) to land it — GitHub's rebase-merge cannot replay a merge-maintained branch. ## Residual Review Findings Multi-agent code review of the PostgreSQL satellite-store ports (U1–U5) applied 3 safe fixes (see `fix(review): apply autofix feedback`). The following are **real but gated** — recorded here as follow-up work rather than auto-applied. All are SQLite→PostgreSQL **concurrency/atomicity regressions**: the sync stores were immune only by SQLite's single-writer, single-threaded-handler execution; the async ports open multi-await read-modify-write windows. **Reachability is low today** because the execution engines that generate concurrent same-run mutations (insight run executor, research orchestrator/dispatcher) are `instanceof`-gated to sync mode in PG. No process-crash class survived (all engine fallbacks correctly guard the sync store). - **[P1] Research `appendResearchEvent` dual-write is non-atomic** (`packages/core/src/async-research-store.ts`, corroborated: adversarial + reliability). The `research_run_events` insert (own transaction) and the `run.events` jsonb update are separate writes — a crash between them, or two concurrent appends, splits the table count from the jsonb array. **Fix:** perform the seq-insert and the jsonb update in one `layer.transactionImmediate`. - **[P1] Research run terminal-reversion via stale full-row persist** (`async-research-store.ts` `persistResearchRun`/`updateResearchStatus`). Concurrent `PATCH /runs/:id/status` + `POST /runs/:id/events` can revert a terminal run to `running` by overwriting the whole row, bypassing the transition guard. **Fix:** scoped column `UPDATE`s with a `WHERE status …` guard, or optimistic version column. - **[P2] `updateResearchRun`/`updateInsightRun` read-then-write TOCTOU** — concurrent PATCHes last-writer-wins on the lifecycle merge. **Fix:** `SELECT … FOR UPDATE` / enclosing transaction. - **[P2] `upsertRun`/`createRunOrThrowConflict` check-then-create race** (`async-insight-store.ts`) — two callers can each create an "active" run. **Fix:** partial unique index on `(projectId, trigger) WHERE status IN ('pending','running')`. - **[P3] `createResearchRetryRun` return-value divergence** — sync returns the pre-update `queued` snapshot; async returns the reloaded `retry_waiting` run (persisted state is identical). Pick one side for cross-backend parity. - **[P2/perf] Mission `getMissionWithHierarchy`/`getMissionHealth` N+1 fan-out** — O(milestones×slices) sequential round-trips hold one pool slot per request; can starve the pool for large hierarchies. **Fix:** batched/joined reads. - **Testing gaps:** no PG-mode concurrency tests (interleaved status/event mutations), no sync↔async parity assertion for the lifecycle-error codes, and no mission status/health rollup parity test vs the sync `MissionStore`. ~~Out of scope (deferred): AI run *execution* (insight/research) + mission autopilot + live SSE mission events remain sync-gated/degraded in PG mode.~~ **Since ported** — insight/research run execution, mission autopilot, and SSE live push all run on the async layer now, which also makes the concurrency findings above genuinely reachable; they remain open follow-ups. --- ## Update — 2026-07-12: production-readiness hardening & live acceptance Everything below landed on this branch since the description above was written: **Production blockers from review — fixed** - `recoverStaleTransitionPending` ported to the async layer (backend moves write + clear the crash-safe marker; startup/maintenance sweeps no longer throw). - Lost-update class fixed: `atomicWriteTaskJson`/`WithAudit` write changed columns only (full-row upserts silently resurrected stale fields across concurrent store instances — the "task stuck unplanned forever" bug). - First-boot **auto-migration**: booting the PG backend over a project with a legacy `fusion.db` migrates it automatically (loud failure, SQLite kept as backup), and the dashboard shows a one-time **"your data was migrated" banner** with the backup paths and a Need-help Discord link. - `pg_dump`/`pg_restore` discovered from common install locations for embedded-mode backups. - The PG suite is part of the blocking merge gate (`test:pg-gate`). **Multi-project isolation (PR #2007, merged into this branch)** - `project_id` partition key on tasks / archived tasks / config, `taskProjectScope` threaded through every scan/claim/count, per-project config rows, layer bound to the project at startup. - Review P1 follow-up: the shared cold-storage `archive.archived_tasks` table is also partitioned and all archived-board reads/counts/searches are scoped. - Schema drift self-heal generalized to schema-qualified columns so existing databases upgrade in place. **Other changes** - Node settings sync **removed** in PG mode (409 `settings-sync-disabled-postgres`) — nodes share state by connecting to the same database; auth sync kept (per-machine file). - Perf (review findings): `listTasks` pushes column filter + ORDER BY + LIMIT/OFFSET into SQL; `getConversation` capped to the most recent 200 messages. - Fixed a false "operator action required" pause-abort log fired on every successfully auto-merged task. **Live acceptance — PASSED (2026-07-12)** A sandboxed instance (isolated HOME, embedded PG, real Opus executor) ran a task through the complete cycle: create → triage (AI spec) → execute → in-review → AI squash-merge landed on the project's `main` → done. A write+read sweep of every data surface (settings, comments, documents, attachments + artifact bridge + artifact edit, chat with real generation, goals, missions, agent mail, secrets, workflows, memory, CC analytics) was green on embedded PG. **Known remaining work** - The per-project `config` PK re-key has no upgrade path for pre-isolation embedded-PG databases (needs a real `DROP CONSTRAINT`/re-key migration; fresh databases are fine). - `pg_dump`/`pg_restore` binaries are not yet bundled in release artifacts (PATH/common-location discovery only). - The satellite-store concurrency findings listed above. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Phil Larson <hello@phillarson.xyz> Co-authored-by: fusion-merge <fusion-merge@local>
Reports Plugin for Fusion
Generates HTML system activity reports with multi-agent review.
Install (one-click)
- Open Settings → Plugins → Fusion Plugins.
- In Bundled Plugins, click Install on Reports.
- Enable the plugin if prompted.
Once installed and enabled, Fusion registers the Reports dashboard destination automatically.
Scaffold seams (interim)
The plugin currently exports four interim scaffold seams to unblock downstream implementation work:
resolveEnabledCadences/ReportsCadence(src/cadence.ts) — interim cadence-resolution seam; scheduled cadence registry + cron/sentinel wiring lands in FN-3779.aggregateReportData+ aggregation types (src/aggregation.ts) — interim aggregation seam; real aggregation orchestration lands in FN-3780.startReportsPipeline+ pipeline dependency interfaces (src/pipeline.ts) — interim orchestrator seam to keep call sites stable while FN-3779/FN-3780 wire real runtime components.createInMemoryReportsRunsStore+ run record/store types (src/runs-store.ts) — interim in-memory run state store; persistent store replacement lands in FN-3784.
Review Panel
The plugin exposes runReviewPanel() / runGeneratedReportReview() to fan out a generated report draft to multiple reviewer agents in parallel.
Panel member settings shape
Each reviewer uses this contract:
{
id: string;
name: string;
perspective: string;
promptTemplateId?: string;
provider?: string;
modelId?: string;
}
perspectiveis appended to the reviewer system prompt.promptTemplateIdselects a template fromsettings.reviewPromptTemplates[templateId]when present.provider+modelIdoptionally override model selection per reviewer.
Prompt template contract
runReviewPanel resolves reviewer templates in this order:
settings.reviewPromptTemplates[promptTemplateId ?? id]settings.reviewPrompt- Built-in fallback (
DEFAULT_REVIEW_PROMPT)
This is the temporary compatibility contract until FN-3782 lands shared review-template helpers.
Individual review shape
{
memberId: string;
memberName: string;
perspective: string;
verdict: "approve" | "revise" | "reject";
summary: string;
highlights: string[];
lowlights: string[];
suggestions: string[];
rawText: string;
durationMs: number;
}
Combined review shape
{
overallVerdict: "approve" | "revise" | "reject";
consensusSummary: string;
mergedHighlights: string[];
mergedLowlights: string[];
mergedSuggestions: string[];
individual: IndividualReview[];
failures: ReviewFailure[];
}
Aggregation is deterministic:
- verdict precedence:
approve < revise < reject - merged arrays are case-insensitive de-duped, first-seen order, max 25 items each
- consensus summary is generated locally from reviewer summaries (no second AI call)
Timeout and failure semantics
- Each reviewer has a hard timeout (
120_000ms). - A single reviewer failure never aborts the full panel.
- Failures are returned as:
{
memberId: string;
reason: "timeout" | "parse_error" | "session_unavailable" | "exception";
message: string;
}
- If all reviewers fail, combined verdict is
rejectwith an explicit consensus summary describing panel failure.
Report Archive
The plugin persists generated reports in SQLite via ensureReportSchema(db) and ReportStore.
Schema
Table: reports
- identity/metadata:
id,cadence,title,metadataJson - period window:
periodStart,periodEnd - lifecycle/status:
status,failureReason - payload references:
draftMarkdown,renderedHtmlPath - review payload:
combinedReviewJson - timestamps:
generationStartedAt,generationCompletedAt,reviewStartedAt,reviewCompletedAt,approvedAt,publishedAt,archivedAt,createdAt,updatedAt - approval actor:
approvedBy
Indexes:
idxReportsCadenceCreatedon(cadence, createdAt DESC, id)idxReportsStatusUpdatedon(status, updatedAt DESC, id)idxReportsPeriodon(periodStart, periodEnd, id)
Status lifecycle
generating → review_pending → review_in_progress → review_complete → approved → published
failed and archived are allowed from any non-terminal state. Idempotent transitions (from === to) are no-ops.
Approval + publish lifecycle (FN-3787)
A parallel approvalState gate now controls human/approver decisions before distribution:
review_complete entry:
approvalRequired=false, autoPublishOnApproval=false→approvalState=approved,status=approvedapprovalRequired=false, autoPublishOnApproval=true→approvalState=published,status=publishedapprovalRequired=true→approvalState=awaiting_approval,status=review_complete
Decision transitions:
awaiting_approval --approve--> approved(or directlypublishedwhenautoPublishOnApproval=true)awaiting_approval --reject--> rejectedapproved --publish--> published
Backfilled legacy rows use approvalState=not_required and are non-actionable.
Authorization rules:
- When
approvalRequired=trueandapproverAgentIdsis non-empty, only listed approver agent IDs may approve/reject/publish. - When
approvalRequired=trueandapproverAgentIds=[], any human dashboard user is allowed; agents are not. publishTargetsrecords publish intent metadata when a report reachespublished.
Share-ready summary blocks (FN-3787)
Approved/published reports can produce deterministic share artifacts via GET /reports/:id/share-blocks:
plainText: compact paste-ready summarymarkdown: heading/bullets + report linkslack: mrkdwn-friendly summaryemailHtml: inline-styled HTML snippet for email clients
share-blocks is intentionally locked (409) until approvalState is approved or published.
Email HTML styling exemption:
emailHtmldeliberately uses inline style attributes and hardcoded hex colors for email-client compatibility; dashboard design-token CSS rules do not apply to this serialized output format.
ReportStore API
createReport(input)getReport(id)listReports(filter?)updateReport(id, patch)setStatus(id, next, opts?)attachReview(id, combinedReview)attachRenderedHtml(id, htmlPath)deleteReport(id)
Emitted events:
report:createdreport:updatedreport:status-changedreport:review-attachedreport:deleted
This archive is the source of truth for downstream report HTML rendering (FN-3785) and dashboard report list/detail flows (FN-3786).
Dashboard view
The plugin registers a primary dashboard view (Reports) via dashboardViews with componentPath: "./dashboard-view".
The view provides:
- History list of reports with filters (cadence, status, period date range, title search, agent filter)
- Embedded detail preview using sandboxed iframe + preview HTML endpoint
- Section quick-jump navigation by stable
data-sectionmarkers - Side-by-side comparison drawer for two reports with section-level diff summary
- Standalone HTML download action wired to the export endpoint