ab228551e9e0fa2e576e65fdca509062c9610629
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
gsxdsm
|
c15c78feeb |
feat: migrate storage from SQLite to PostgreSQL (#1793)
# Migrate storage from SQLite to PostgreSQL — full dashboard cutover Migrates Fusion's storage layer to the embedded PostgreSQL `AsyncDataLayer` (the default backend) and **completes the satellite-store + feature cutover** so every dashboard and Command Center surface works in PG mode. ## Status — every surface works in embedded-PG mode Verified live against a running embedded-Postgres dashboard (all **200**, zero 5xx) and gate-tested (**23 files / 99 tests** on embedded PG, plus engine-core 294 and ci-shape 63 in the blocking merge gate; core/engine/cli/dashboard typecheck clean). | Area | Surfaces | State | |---|---|---| | Satellite stores | workflows, todos, insights, research, missions, goals, mailbox | ✅ | | Views | artifacts, documents, evals | ✅ | | Command Center | activity, productivity, team, tokens, tools, **workflows**, **github**, **signals**, **plugin-activations**, **live** (all 10) | ✅ | | Run execution | insight generation, research run execution | ✅ (store-path; AI step needs a provider) | | Live updates | SSE push for mission/research/insight events | ✅ | | Workflow editing | create / update / delete / select (+ id counter) | ✅ | | Engine | mission autopilot, incident-signal ingestion, regression storm-guard, agent wake-on-message | ✅ | | Core | tasks, agents, secrets, automations, memory, chat, usage, PRs, git | ✅ | ## Approach Each satellite store gets an `Async<Store>` wrapper exposing the sync store's method names over the existing `async-*-store.ts` helpers; `get<Store>Store()` returns a `Sync | Async` union; consumers `await` (harmless on sync), and engine/CLI paths that can't convert use `instanceof Sync` graceful fallback. Analytics aggregators branch on `"ping" in dbOrLayer` to run schema-qualified raw SQL over `project.*` (snake_case) in PG. Executors/orchestrators/autopilot are await-converted to drive the union store; the async store wrappers extend `EventEmitter` so SSE live-push fires in both backends. Not-yet-ported capabilities degrade gracefully (never 500) and are individually called out in commits. ## Sync with main The branch is kept continuously merged with `main` (currently through FN-7845, 2026-07-12); the earlier "final rebase deferred" note no longer applies. Use **Create a merge commit** (or squash) to land it — GitHub's rebase-merge cannot replay a merge-maintained branch. ## Residual Review Findings Multi-agent code review of the PostgreSQL satellite-store ports (U1–U5) applied 3 safe fixes (see `fix(review): apply autofix feedback`). The following are **real but gated** — recorded here as follow-up work rather than auto-applied. All are SQLite→PostgreSQL **concurrency/atomicity regressions**: the sync stores were immune only by SQLite's single-writer, single-threaded-handler execution; the async ports open multi-await read-modify-write windows. **Reachability is low today** because the execution engines that generate concurrent same-run mutations (insight run executor, research orchestrator/dispatcher) are `instanceof`-gated to sync mode in PG. No process-crash class survived (all engine fallbacks correctly guard the sync store). - **[P1] Research `appendResearchEvent` dual-write is non-atomic** (`packages/core/src/async-research-store.ts`, corroborated: adversarial + reliability). The `research_run_events` insert (own transaction) and the `run.events` jsonb update are separate writes — a crash between them, or two concurrent appends, splits the table count from the jsonb array. **Fix:** perform the seq-insert and the jsonb update in one `layer.transactionImmediate`. - **[P1] Research run terminal-reversion via stale full-row persist** (`async-research-store.ts` `persistResearchRun`/`updateResearchStatus`). Concurrent `PATCH /runs/:id/status` + `POST /runs/:id/events` can revert a terminal run to `running` by overwriting the whole row, bypassing the transition guard. **Fix:** scoped column `UPDATE`s with a `WHERE status …` guard, or optimistic version column. - **[P2] `updateResearchRun`/`updateInsightRun` read-then-write TOCTOU** — concurrent PATCHes last-writer-wins on the lifecycle merge. **Fix:** `SELECT … FOR UPDATE` / enclosing transaction. - **[P2] `upsertRun`/`createRunOrThrowConflict` check-then-create race** (`async-insight-store.ts`) — two callers can each create an "active" run. **Fix:** partial unique index on `(projectId, trigger) WHERE status IN ('pending','running')`. - **[P3] `createResearchRetryRun` return-value divergence** — sync returns the pre-update `queued` snapshot; async returns the reloaded `retry_waiting` run (persisted state is identical). Pick one side for cross-backend parity. - **[P2/perf] Mission `getMissionWithHierarchy`/`getMissionHealth` N+1 fan-out** — O(milestones×slices) sequential round-trips hold one pool slot per request; can starve the pool for large hierarchies. **Fix:** batched/joined reads. - **Testing gaps:** no PG-mode concurrency tests (interleaved status/event mutations), no sync↔async parity assertion for the lifecycle-error codes, and no mission status/health rollup parity test vs the sync `MissionStore`. ~~Out of scope (deferred): AI run *execution* (insight/research) + mission autopilot + live SSE mission events remain sync-gated/degraded in PG mode.~~ **Since ported** — insight/research run execution, mission autopilot, and SSE live push all run on the async layer now, which also makes the concurrency findings above genuinely reachable; they remain open follow-ups. --- ## Update — 2026-07-12: production-readiness hardening & live acceptance Everything below landed on this branch since the description above was written: **Production blockers from review — fixed** - `recoverStaleTransitionPending` ported to the async layer (backend moves write + clear the crash-safe marker; startup/maintenance sweeps no longer throw). - Lost-update class fixed: `atomicWriteTaskJson`/`WithAudit` write changed columns only (full-row upserts silently resurrected stale fields across concurrent store instances — the "task stuck unplanned forever" bug). - First-boot **auto-migration**: booting the PG backend over a project with a legacy `fusion.db` migrates it automatically (loud failure, SQLite kept as backup), and the dashboard shows a one-time **"your data was migrated" banner** with the backup paths and a Need-help Discord link. - `pg_dump`/`pg_restore` discovered from common install locations for embedded-mode backups. - The PG suite is part of the blocking merge gate (`test:pg-gate`). **Multi-project isolation (PR #2007, merged into this branch)** - `project_id` partition key on tasks / archived tasks / config, `taskProjectScope` threaded through every scan/claim/count, per-project config rows, layer bound to the project at startup. - Review P1 follow-up: the shared cold-storage `archive.archived_tasks` table is also partitioned and all archived-board reads/counts/searches are scoped. - Schema drift self-heal generalized to schema-qualified columns so existing databases upgrade in place. **Other changes** - Node settings sync **removed** in PG mode (409 `settings-sync-disabled-postgres`) — nodes share state by connecting to the same database; auth sync kept (per-machine file). - Perf (review findings): `listTasks` pushes column filter + ORDER BY + LIMIT/OFFSET into SQL; `getConversation` capped to the most recent 200 messages. - Fixed a false "operator action required" pause-abort log fired on every successfully auto-merged task. **Live acceptance — PASSED (2026-07-12)** A sandboxed instance (isolated HOME, embedded PG, real Opus executor) ran a task through the complete cycle: create → triage (AI spec) → execute → in-review → AI squash-merge landed on the project's `main` → done. A write+read sweep of every data surface (settings, comments, documents, attachments + artifact bridge + artifact edit, chat with real generation, goals, missions, agent mail, secrets, workflows, memory, CC analytics) was green on embedded PG. **Known remaining work** - The per-project `config` PK re-key has no upgrade path for pre-isolation embedded-PG databases (needs a real `DROP CONSTRAINT`/re-key migration; fresh databases are fine). - `pg_dump`/`pg_restore` binaries are not yet bundled in release artifacts (PATH/common-location discovery only). - The satellite-store concurrency findings listed above. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Phil Larson <hello@phillarson.xyz> Co-authored-by: fusion-merge <fusion-merge@local> |
||
|
gsxdsm
|
6ea53966f6 |
FN-7885: migrate remaining direct clipboard callers to shared copyTextToClipboard helper
Replaces the last direct navigator.clipboard.writeText() call sites across dashboard components and the reports plugin with the shared copyTextToClipboard helper, fixing copy actions that crashed or silently failed on non-secure origins (HTTP/mobile). - Migrated AgentDetailView, AgentErrorDetailsModal, CliBinaryPanel, GitManagerModal, LoginInstructions, PrPanel, SecretsView, and StashConflictModal to use copyTextToClipboard (secure-context guard + execCommand fallback, boolean result handling) instead of calling navigator.clipboard directly. - Migrated the fusion-plugin-reports ShareBlocksPanel to the same helper and added a vitest alias so the plugin's subpath import resolves to the dashboard's copyToClipboard util instead of collapsing to its package root. - Added ./app/utils/copyToClipboard subpath export to @fusion/dashboard's package.json. - Added/extended tests covering copy success, fallback, and failure paths for AgentDetailView, CliBinaryPanel, AgentErrorDetailsModal, GitManagerModal, SecretsView, StashConflictModal, and ShareBlocksPanel. - Added a patch changeset documenting the fix for @runfusion/fusion. Files changed: .changeset/fn-7885-clipboard-migration.md | 7 ++ packages/dashboard/app/components/AgentDetailView.tsx | 16 ++++- packages/dashboard/app/components/AgentErrorDetailsModal.tsx | 5 +- packages/dashboard/app/components/CliBinaryPanel.tsx | 16 +++-- packages/dashboard/app/components/GitManagerModal.tsx | 16 ++++- packages/dashboard/app/components/LoginInstructions.tsx | 21 +++--- packages/dashboard/app/components/PrPanel.tsx | 9 ++- packages/dashboard/app/components/SecretsView.tsx | 11 ++- packages/dashboard/app/components/StashConflictModal.tsx | 13 ++-- packages/dashboard/app/components/__tests__/AgentDetailView.copy.test.tsx | 56 +++++++++++++++ packages/dashboard/app/components/__tests__/AgentErrorDetailsModal.test.tsx | 18 +++++ packages/dashboard/app/components/__tests__/CliBinaryPanel.copy.test.tsx | 68 ++++++++++++++++++ packages/dashboard/app/components/__tests__/GitManagerModal.test.tsx | 23 ++++++ packages/dashboard/app/components/__tests__/SecretsView.test.tsx | 82 ++++++++++++++++++++++ packages/dashboard/app/components/__tests__/StashConflictModal.test.tsx | 25 ++++++- packages/dashboard/package.json | 4 ++ plugins/fusion-plugin-reports/src/dashboard/components/ShareBlocksPanel.tsx | 5 +- plugins/fusion-plugin-reports/src/dashboard/components/__tests__/ShareBlocksPanel.test.tsx | 43 +++++++++++- plugins/fusion-plugin-reports/vitest.config.ts | 2 + 19 files changed, 402 insertions(+), 38 deletions(-) Fusion-Task-Id: FN-7885 Fusion-Task-Lineage: 122a54ea-962b-4ae1-98ac-c28e03f4f8ca Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai> |
||
|
gsxdsm
|
8bc3d7b0a5 |
FN-6042: raise dependency security floors
Harden dependency floors and update the Vitest toolchain to patched releases. - upgrade workspace vitest and @vitest/coverage-v8 dependencies to the 4.1 line across packages and plugins - pin transitive protobufjs via pnpm overrides and lockfile updates to patched versions - adapt Vitest configs, engine test helpers, and security-floor coverage for the new dependency baselines - add the published CLI changeset and related workspace/package metadata updates included in the task branch Files changed: .changeset/fn-6042-security-dependencies.md | 5 + AGENTS.md | 4 + Dockerfile | 3 + docs/PLUGIN_AUTHORING.md | 2 - package.json | 3 +- packages/cli/package.json | 4 +- packages/cli/vitest.config.ts | 2 +- packages/core/package.json | 4 +- packages/core/vitest.config.ts | 2 +- packages/dashboard/app/test/mockApi.ts | 4 +- packages/dashboard/package.json | 4 +- packages/dashboard/vitest.config.ts | 2 +- packages/desktop/package.json | 4 +- packages/desktop/vitest.config.ts | 2 +- packages/droid-cli/package.json | 2 +- packages/droid-cli/vitest.config.ts | 2 +- packages/engine/package.json | 4 +- .../engine/src/__tests__/executor-test-helpers.ts | 29 +- .../engine/src/__tests__/gridlock-detector.test.ts | 5 +- .../src/__tests__/heartbeat-scheduler.test.ts | 3 +- packages/engine/src/__tests__/scheduler.test.ts | 24 +- packages/engine/src/__tests__/self-healing.test.ts | 5 + packages/engine/tsconfig.json | 3 +- packages/engine/vitest.config.ts | 10 +- packages/i18n/package.json | 2 +- packages/i18n/vitest.config.ts | 7 + packages/mobile/package.json | 2 +- packages/mobile/vitest.config.ts | 2 +- packages/pi-claude-cli/package.json | 2 +- packages/pi-claude-cli/vitest.config.ts | 2 +- packages/pi-llama-cpp/package.json | 2 +- packages/pi-llama-cpp/vitest.config.ts | 2 +- packages/plugin-sdk/package.json | 2 +- packages/plugin-sdk/vitest.config.ts | 2 +- .../examples/fusion-plugin-auto-label/package.json | 2 +- .../fusion-plugin-auto-label/vitest.config.ts | 2 +- .../examples/fusion-plugin-ci-status/package.json | 2 +- .../fusion-plugin-ci-status/vitest.config.ts | 2 +- .../fusion-plugin-notification/package.json | 2 +- .../fusion-plugin-notification/vitest.config.ts | 2 +- .../fusion-plugin-settings-demo/package.json | 2 +- .../fusion-plugin-settings-demo/vitest.config.ts | 2 +- plugins/fusion-plugin-acp-runtime/package.json | 2 +- plugins/fusion-plugin-acp-runtime/vitest.config.ts | 2 +- plugins/fusion-plugin-agent-browser/package.json | 2 +- .../fusion-plugin-agent-browser/vitest.config.ts | 2 +- .../fusion-plugin-cli-printing-press/package.json | 2 +- .../vitest.config.ts | 2 +- .../package.json | 2 +- .../src/__tests__/orchestrator-live-output.test.ts | 4 +- .../vitest.config.ts | 2 +- plugins/fusion-plugin-cursor-runtime/package.json | 4 +- .../fusion-plugin-dependency-graph/package.json | 4 +- .../vitest.config.ts | 2 +- plugins/fusion-plugin-droid-runtime/package.json | 2 +- .../fusion-plugin-droid-runtime/vitest.config.ts | 2 +- plugins/fusion-plugin-even-cards/package.json | 2 +- plugins/fusion-plugin-even-cards/vitest.config.ts | 2 +- .../package.json | 2 +- .../vitest.config.ts | 2 +- plugins/fusion-plugin-hermes-runtime/package.json | 2 +- .../fusion-plugin-hermes-runtime/vitest.config.ts | 2 +- .../fusion-plugin-openclaw-runtime/package.json | 2 +- .../vitest.config.ts | 2 +- .../fusion-plugin-paperclip-runtime/package.json | 2 +- .../vitest.config.ts | 2 +- plugins/fusion-plugin-reports/package.json | 2 +- .../src/__tests__/review-panel.test.ts | 6 +- plugins/fusion-plugin-reports/vitest.config.ts | 2 +- plugins/fusion-plugin-roadmap/package.json | 2 +- plugins/fusion-plugin-roadmap/vitest.config.ts | 2 +- plugins/fusion-plugin-whatsapp-chat/package.json | 2 +- .../fusion-plugin-whatsapp-chat/vitest.config.ts | 2 +- pnpm-lock.yaml | 626 ++++++++------------- .../__tests__/dependency-security-floor.test.mjs | 95 ++++ 75 files changed, 475 insertions(+), 491 deletions(-) Fusion-Task-Id: FN-6042 Fusion-Task-Lineage: fff6a1cb-8937-435c-9a91-b7c7a59cc80e |
||
|
Josemi Liebana
|
dc0b764977 |
fix(test): address review — inherit coreSetup from root via extends: true
Remove duplicate coreSetup from reports-dashboard project setupFiles. With extends: true, coreSetup is already inherited from the root config. Only dashboardSetup needs to be explicitly added at the project level. |
||
|
Josemi Liebana
|
6fe7d6909c |
fix(test): migrate fusion-plugin-reports vitest config from deprecated environmentMatchGlobs to test.projects
Resolves CI failures where Node.js 24 caused ReferenceError: window is not defined in non-dashboard tests that shared setupFiles with dashboard tests. Follows the same pattern already used by fusion-plugin-roadmap: split into two vitest projects with proper environment isolation: - reports-dashboard: jsdom, includes src/dashboard/** tests + dashboard test-setup.ts - reports-node: node, includes all other tests, excludes dashboard tests No test changes — same 104 tests, same pass/fail state. |
||
|
Fusion
|
43d3ac2c18 |
feat(FN-3786): add reports dashboard view and comparison tooling
Adds a Reports dashboard view (`ReportsView`) to the reports plugin with a full supporting stack of components (`ReportComparisonDrawer`, `ReportDetailPanel`, `ReportFiltersBar`, `ReportListItem`, `ReportEmptyState`), hooks (`useReports`, `useReportPreview`, `useReportSectionDiff`, `useViewportMode` Fusion-Task-Id: FN-3786 |
||
|
Fusion
|
d71c9102f7 |
feat(FN-3778): add reports plugin scaffold with notification service improv
The merge introduces a new `fusion-plugin-reports` plugin scaffold with settings schema, manifest, entry point, and tests, alongside a mobile session switcher in the chat header. It also adds a mailbox notifications system with ntfy/webhook providers, cleans up roadmap types from core, and includes Fusion-Task-Id: FN-3778 |