## Summary
Speeds up **time-to-HTTP-ready** for `fn dashboard` and `fn serve` after
the PostgreSQL cutover without reintroducing the historical 3s
cwd-engine race that degraded webhooks.
- **Dashboard store share (serve parity):** inject the factory-booted
`TaskStore` as `externalTaskStore` so cwd `ensureEngine` does not open a
second pool; share only when store root matches project working
directory (multi-project safe).
- **Serve multi-project:** stop awaiting `startAll()` before listen;
await only the primary engine; background the rest + reconciliation.
- **Defer non-route-critical engine work:** ordered OAuth (refresh →
monitor), automation schedule syncs, and auto-merge **enqueue** after
the engine handle is returnable.
- **Critical-path merge status clear:** still clear stale
`merging`/`merging-pr` before ready so manual merge is not blocked after
crash.
- **Serve `--paused`:** apply `enginePaused` before
`ensureEngine`/`startAll` (dashboard ordering).
- **Stop safety:** generation counter so deferred tails cannot resume
after `stop()` clears `shuttingDown`.
- **Phase timing:** shared `phaseTime` helper, factory substep logs,
serve time-to-listen.
Plan: `docs/plans/2026-07-14-001-feat-faster-startup-plan.md`
## Test plan
- [x] `packages/engine` — `project-engine-manager.test.ts` (path-matched
external store)
- [x] `packages/engine` — `project-engine-deferred-startup.test.ts`
(status clear, OAuth order, stop generation)
- [x] `packages/cli` — `startup-phase.test.ts`
- [x] `packages/cli` — `serve.test.ts` (60 tests, including `--paused`)
- [ ] Local: warm `fn dashboard` / `fn serve` and compare `startup phase
*` / `time-to-listen` logs
- [ ] `pnpm smoke:boot` (real serve `/api/health` on ephemeral port)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance**
* Improved dashboard and serve startup times, including faster
time-to-listen and time-to-ready.
* Moved non-essential background initialization off the critical startup
path.
* Parallelized dashboard service initialization where possible.
* **Reliability**
* Improved multi-project startup handling and project selection.
* Prevented cross-project task-store sharing.
* Added safer shutdown behavior for partially completed startup.
* **Diagnostics**
* Added startup phase timing logs to help identify performance
bottlenecks.
* **Tests**
* Expanded coverage for deferred startup, shutdown, project isolation,
and startup timing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Engine and dashboard traffic now stays on the authoritative PostgreSQL
layer across execution, recovery, project discovery, planning sessions,
analytics, and shutdown. The dashboard no longer presents a migration
notice for a cutover that is already mandatory.
## Design decisions
- Runtime composition requires an async data layer instead of
constructing a hidden SQLite fallback.
- Engine workflow, mission, claim, and self-healing reads await their
PostgreSQL-backed store contracts.
- Project-scoped dashboard stores retain and close their backend owner
exactly once.
- The dashboard test quarantine entry remains paired with its Vitest
exclusion, preserving the repository’s deletion-ratchet policy.
## Validation
- Core, Engine, Dashboard, CLI, and Desktop typechecks pass on the
stacked branch.
- `pnpm test:gate` passes all 478 gate tests.
- This PR changes 62 files.
## Stack
- Depends on #2108.
- CLI/desktop/ops, plugins, and docs/release follow in later PRs.
Related: #2105
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Project discovery now recognizes projects using the
`.fusion/project.json` marker.
* Knowledge indexing and search are more reliable across project-scoped
storage.
* **Bug Fixes**
* Improved session, audit timeline, approval, monitoring, and analytics
data consistency.
* Prevented stale planning-session updates and project-store shutdown
races.
* Ensured chat usage and CLI session status are saved before continuing.
* **UI Changes**
* Removed the storage migration notice banner now that the PostgreSQL
transition is complete.
* **Reliability**
* Improved shutdown handling, workflow execution, and worktree behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
# Migrate storage from SQLite to PostgreSQL — full dashboard cutover
Migrates Fusion's storage layer to the embedded PostgreSQL
`AsyncDataLayer` (the default backend) and **completes the
satellite-store + feature cutover** so every dashboard and Command
Center surface works in PG mode.
## Status — every surface works in embedded-PG mode
Verified live against a running embedded-Postgres dashboard (all
**200**, zero 5xx) and gate-tested (**23 files / 99 tests** on embedded
PG, plus engine-core 294 and ci-shape 63 in the blocking merge gate;
core/engine/cli/dashboard typecheck clean).
| Area | Surfaces | State |
|---|---|---|
| Satellite stores | workflows, todos, insights, research, missions,
goals, mailbox | ✅ |
| Views | artifacts, documents, evals | ✅ |
| Command Center | activity, productivity, team, tokens, tools,
**workflows**, **github**, **signals**, **plugin-activations**, **live**
(all 10) | ✅ |
| Run execution | insight generation, research run execution | ✅
(store-path; AI step needs a provider) |
| Live updates | SSE push for mission/research/insight events | ✅ |
| Workflow editing | create / update / delete / select (+ id counter) |
✅ |
| Engine | mission autopilot, incident-signal ingestion, regression
storm-guard, agent wake-on-message | ✅ |
| Core | tasks, agents, secrets, automations, memory, chat, usage, PRs,
git | ✅ |
## Approach
Each satellite store gets an `Async<Store>` wrapper exposing the sync
store's method names over the existing `async-*-store.ts` helpers;
`get<Store>Store()` returns a `Sync | Async` union; consumers `await`
(harmless on sync), and engine/CLI paths that can't convert use
`instanceof Sync` graceful fallback. Analytics aggregators branch on
`"ping" in dbOrLayer` to run schema-qualified raw SQL over `project.*`
(snake_case) in PG. Executors/orchestrators/autopilot are
await-converted to drive the union store; the async store wrappers
extend `EventEmitter` so SSE live-push fires in both backends.
Not-yet-ported capabilities degrade gracefully (never 500) and are
individually called out in commits.
## Sync with main
The branch is kept continuously merged with `main` (currently through
FN-7845, 2026-07-12); the earlier "final rebase deferred" note no longer
applies. Use **Create a merge commit** (or squash) to land it — GitHub's
rebase-merge cannot replay a merge-maintained branch.
## Residual Review Findings
Multi-agent code review of the PostgreSQL satellite-store ports (U1–U5)
applied 3 safe fixes (see `fix(review): apply autofix feedback`). The
following are **real but gated** — recorded here as follow-up work
rather than auto-applied. All are SQLite→PostgreSQL
**concurrency/atomicity regressions**: the sync stores were immune only
by SQLite's single-writer, single-threaded-handler execution; the async
ports open multi-await read-modify-write windows. **Reachability is low
today** because the execution engines that generate concurrent same-run
mutations (insight run executor, research orchestrator/dispatcher) are
`instanceof`-gated to sync mode in PG. No process-crash class survived
(all engine fallbacks correctly guard the sync store).
- **[P1] Research `appendResearchEvent` dual-write is non-atomic**
(`packages/core/src/async-research-store.ts`, corroborated: adversarial
+ reliability). The `research_run_events` insert (own transaction) and
the `run.events` jsonb update are separate writes — a crash between
them, or two concurrent appends, splits the table count from the jsonb
array. **Fix:** perform the seq-insert and the jsonb update in one
`layer.transactionImmediate`.
- **[P1] Research run terminal-reversion via stale full-row persist**
(`async-research-store.ts` `persistResearchRun`/`updateResearchStatus`).
Concurrent `PATCH /runs/:id/status` + `POST /runs/:id/events` can revert
a terminal run to `running` by overwriting the whole row, bypassing the
transition guard. **Fix:** scoped column `UPDATE`s with a `WHERE status
…` guard, or optimistic version column.
- **[P2] `updateResearchRun`/`updateInsightRun` read-then-write TOCTOU**
— concurrent PATCHes last-writer-wins on the lifecycle merge. **Fix:**
`SELECT … FOR UPDATE` / enclosing transaction.
- **[P2] `upsertRun`/`createRunOrThrowConflict` check-then-create race**
(`async-insight-store.ts`) — two callers can each create an "active"
run. **Fix:** partial unique index on `(projectId, trigger) WHERE status
IN ('pending','running')`.
- **[P3] `createResearchRetryRun` return-value divergence** — sync
returns the pre-update `queued` snapshot; async returns the reloaded
`retry_waiting` run (persisted state is identical). Pick one side for
cross-backend parity.
- **[P2/perf] Mission `getMissionWithHierarchy`/`getMissionHealth` N+1
fan-out** — O(milestones×slices) sequential round-trips hold one pool
slot per request; can starve the pool for large hierarchies. **Fix:**
batched/joined reads.
- **Testing gaps:** no PG-mode concurrency tests (interleaved
status/event mutations), no sync↔async parity assertion for the
lifecycle-error codes, and no mission status/health rollup parity test
vs the sync `MissionStore`.
~~Out of scope (deferred): AI run *execution* (insight/research) +
mission autopilot + live SSE mission events remain sync-gated/degraded
in PG mode.~~ **Since ported** — insight/research run execution, mission
autopilot, and SSE live push all run on the async layer now, which also
makes the concurrency findings above genuinely reachable; they remain
open follow-ups.
---
## Update — 2026-07-12: production-readiness hardening & live acceptance
Everything below landed on this branch since the description above was
written:
**Production blockers from review — fixed**
- `recoverStaleTransitionPending` ported to the async layer (backend
moves write + clear the crash-safe marker; startup/maintenance sweeps no
longer throw).
- Lost-update class fixed: `atomicWriteTaskJson`/`WithAudit` write
changed columns only (full-row upserts silently resurrected stale fields
across concurrent store instances — the "task stuck unplanned forever"
bug).
- First-boot **auto-migration**: booting the PG backend over a project
with a legacy `fusion.db` migrates it automatically (loud failure,
SQLite kept as backup), and the dashboard shows a one-time **"your data
was migrated" banner** with the backup paths and a Need-help Discord
link.
- `pg_dump`/`pg_restore` discovered from common install locations for
embedded-mode backups.
- The PG suite is part of the blocking merge gate (`test:pg-gate`).
**Multi-project isolation (PR #2007, merged into this branch)**
- `project_id` partition key on tasks / archived tasks / config,
`taskProjectScope` threaded through every scan/claim/count, per-project
config rows, layer bound to the project at startup.
- Review P1 follow-up: the shared cold-storage `archive.archived_tasks`
table is also partitioned and all archived-board reads/counts/searches
are scoped.
- Schema drift self-heal generalized to schema-qualified columns so
existing databases upgrade in place.
**Other changes**
- Node settings sync **removed** in PG mode (409
`settings-sync-disabled-postgres`) — nodes share state by connecting to
the same database; auth sync kept (per-machine file).
- Perf (review findings): `listTasks` pushes column filter + ORDER BY +
LIMIT/OFFSET into SQL; `getConversation` capped to the most recent 200
messages.
- Fixed a false "operator action required" pause-abort log fired on
every successfully auto-merged task.
**Live acceptance — PASSED (2026-07-12)**
A sandboxed instance (isolated HOME, embedded PG, real Opus executor)
ran a task through the complete cycle: create → triage (AI spec) →
execute → in-review → AI squash-merge landed on the project's `main` →
done. A write+read sweep of every data surface (settings, comments,
documents, attachments + artifact bridge + artifact edit, chat with real
generation, goals, missions, agent mail, secrets, workflows, memory, CC
analytics) was green on embedded PG.
**Known remaining work**
- The per-project `config` PK re-key has no upgrade path for
pre-isolation embedded-PG databases (needs a real `DROP
CONSTRAINT`/re-key migration; fresh databases are fine).
- `pg_dump`/`pg_restore` binaries are not yet bundled in release
artifacts (PATH/common-location discovery only).
- The satellite-store concurrency findings listed above.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Phil Larson <hello@phillarson.xyz>
Co-authored-by: fusion-merge <fusion-merge@local>
Adds a best-effort, idempotent dashboard inbox notice announcing the upcoming embedded-Postgres storage migration, delivered once per project on the first engine start under the Fusion 0.59.x release line.
- New `deliverPostgresMigrationNoticeIfNeeded` in `@fusion/engine` (`postgres-migration-notice.ts`) builds and sends a `system` -> `user` inbox message via `MessageStore`, gated to version `0.59.x` by `isPostgresMigrationNoticeVersion`
- Idempotency via existing inbox message `metadata.kind = "postgres-migration-notice"` marker (no new settings key or table), so restarts never duplicate the notice
- Delivery is fully best-effort: any `MessageStore` failure is caught, logged as a warning, and never blocks or fails `ProjectEngine.start()`
- `ProjectEngine.start()` invokes the notice after runtime start, using an injected `cliPackageVersion` threaded from the CLI layer through `EngineManagerOptions` / `ProjectEngineOptions` so the engine never imports CLI/dashboard code directly
- `daemon.ts`, `dashboard.ts`, and `serve.ts` resolve the published `@runfusion/fusion` version via `getCliPackageVersion` / `isUnresolvedCliPackageVersion` and pass it into `ProjectEngineManager`
- Exported new symbols (`POSTGRES_MIGRATION_HELP_URL`, `POSTGRES_MIGRATION_NOTICE_KIND`, `deliverPostgresMigrationNoticeIfNeeded`, `isPostgresMigrationNoticeVersion`, related types) from `@fusion/engine`, and `isUnresolvedCliPackageVersion` from `@fusion/dashboard`
- New unit tests covering version matching and single-delivery/idempotency behavior
- Docs updated (`docs/agents.md`, `docs/dashboard-guide.md`) to describe the one-time notice and its dedup key
- Changeset added for `@runfusion/fusion` (minor, feature)
Files changed:
.changeset/fn-7879-postgres-migration-inbox-notice.md | 7 ++
docs/agents.md | 1 +
docs/dashboard-guide.md | 1 +
packages/cli/src/commands/daemon.ts | 6 +-
packages/cli/src/commands/dashboard.ts | 5 +
packages/cli/src/commands/serve.ts | 6 +-
packages/dashboard/src/index.ts | 2 +-
packages/engine/src/__tests__/postgres-migration-notice.test.ts | 140 +++++++++++++++++++++
packages/engine/src/index.ts | 9 ++
packages/engine/src/postgres-migration-notice.ts | 107 ++++++++++++++++
packages/engine/src/project-engine-manager.ts | 6 +
packages/engine/src/project-engine.ts | 12 ++
12 files changed, 299 insertions(+), 3 deletions(-)
Fusion-Task-Id: FN-7879
Fusion-Task-Lineage: 201877e5-6bdc-4168-a8ac-ae0e50ec8308
Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
Move externalEngines.delete(projectId) to immediately after acquiring the
singleton lock instead of after engine.start() succeeds. If a project was
marked external, the holder exits, acquire succeeds, but start() then throws,
the success-path delete never ran and hasRunningEngine() reported a phantom
engine forever. Added a regression test for the failed-takeover path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Stop reconciliation/startAll/onProjectAccessed from warning every tick for
externally-owned engines: swallow EngineAlreadyRunningError in the outer
catches (it's expected and already logged once in createAndStart)
- Add FNXC:DashboardHealth requirement-trace comments on the externalEngines
field and the dashboard hasRunningEngine health check
- Add regression test: reconciliation stays quiet across ticks for an
externally-owned engine (inner refusal logged once, outer failure suppressed)
- Add regression test: hasDashboardEngine legacy fallback to getAllEngines when
hasRunningEngine is unavailable on the manager
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The dashboard's engine-availability health check only counted engines
this process started. A second launch (e.g. `pnpm dev dashboard`
alongside an already-running `fusion`) is correctly refused the
per-machine engine singleton lock, so its engine map stays empty and
the dashboard showed a false "engine not running" banner even though an
engine was live on the machine.
ProjectEngineManager now records projects whose singleton lock is held
by another process (via EngineAlreadyRunningError) and exposes
hasRunningEngine(), which the health endpoint consults so the banner
reflects machine-level truth. Reconciliation still retries so this
process takes over if the other exits, and the "refusing to start" log
fires once per project instead of on every 30s reconciliation tick.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds PrReconciler — a per-repo, self-owned polling loop (started from the
runtime layer in project-engine.ts, NOT the scheduler) that ETag-probes
GitHub, deep-fetches on change, persists mirror state, clears unverified
on first reconcile, and fires releaseHeldTaskByEvent(github:pr-<event>)
for transitions (changes-requested/approved/conflict/conflict-cleared/
merged/closed). Drops terminal entities; persists an audit event on error.
GitHub ops injected via PrReconcileGithubOps at the 3 CLI sites; engine
never imports the dashboard client. scheduler.ts stays PR-free (R20),
pinned by a regression test. 8 tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the three first-class PR workflow node kinds and their handlers via
createPrNodeHandlers(deps), registered in createDefaultNodeHandlers
(fail-closed pr-nodes-unwired when absent). GitHub ops are injected as
callbacks (PrNodeGithubOps) at all three CLI sites (daemon/serve/dashboard)
so the engine never imports the dashboard client (FN-3049). pr-create
routes open/failed as outcomes; pr-merge passes expectedHeadOid and never
writes 'merged' (reconcile corroborates); pr-respond delegates to an
injected respond callback (U5 fills the body). 10 tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Push the single group PR's body (member checklist, x/N landed) on each member
landing via an injected SyncGroupPrFn — new updatePr/closePr GitHubClient
helpers (gh CLI + API parity); refreshPrInBackground is task-scoped/wrong
direction and intentionally not reused. Sync failures are non-fatal+retryable;
out-of-band closed/merged PRs reconcile prState instead of erroring. New
POST /branch-groups/:id/abandon closes the PR best-effort and marks the group
abandoned. Also fixes the U5-introduced stub-context regression in the U4
dashboard bridge test (missing options).
Group promotion in PR mode previously flipped prState to 'open' without ever
calling GitHub — prNumber/prUrl were never populated. Add an injected
CreateGroupPrFn (mirrors the processPullRequestMerge seam, no engine→dashboard
import): coordinator creates-or-reuses exactly one PR per group, persists
prNumber/prUrl/prState, and leaves state untouched on GitHub failure so
re-promotion retries. Idempotent via persisted prNumber +
getBranchGroupByBranchName. Wired at all three CLI engine-construction sites
(daemon/dashboard/serve).
Adds a per-machine singleton lock that engages in
ProjectEngineManager.createAndStart() before any engine subsystems
spin up. Two fn dashboard processes can no longer run engines for the
same project on the same host — previously they would share .fusion/
state and corrupt worktrees / task rows for in-process projects.
The guard combines two independent checks:
- A proper-lockfile file at <project>/.fusion/engine.lock with
stale-lock recovery (auto-released on process death).
- A loopback listener on a hashed per-project address — UDS on
POSIX, named pipe on Windows. Stale UDS files are probed and
unlinked before a retry bind.
Failures raise EngineAlreadyRunningError. Both guards are released
from stopAll() and pauseProject(); a release on engine.start()
failure lets retries re-acquire cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add canonical working-directory resolution with node mapping support in core and engine paths
- Update runtime/multi-project documentation and include published-package changesets for the merged work
- Migrate roadmap dashboard surface to the bundled plugin registry flow and adjust lazy-view integration/tests
- Default non-ephemeral agents to active state and extend related agent/route/executor test coverage
Fusion-Task-Id: FN-3508
- Add pauseProject() and resumeProject() methods to ProjectEngineManager
- Wire pause and resume routes to engineManager with proper error handling
- Update frontend useProjectActions hook to use dedicated pause/resume APIs
- Add comprehensive tests for pause/resume in ProjectEngineManager
- Add route tests for project pause/resume with engineManager mocks
- Fix mock stubs for getProject/updateProject/updateProjectHealth
- ProjectEngineManager.startReconciliation() polls for newly registered
projects every 30s and starts their engines without requiring UI access
- Expose global concurrency limit in dashboard settings
- Fix SettingsModal test cleanup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ProjectEngineManager now creates a single AgentSemaphore and injects it
into all engines via config.globalSemaphore. Previously each engine created
its own semaphore, so the globalMaxConcurrent limit was not enforced across
projects. The semaphore dynamically reads the limit and listens for
concurrency:changed events for live updates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- docs(FN-1626): document PWA home bar gap token and updated bottom-layout contract
- fix(pre-existing): add missing invalidateAllGlobalSettingsCaches export to unblock build
- feat(FN-1626): update regression tests for standalone spacing and PWA home bar gap
- feat(FN-1626): add PWA home bar gap token and update bottom-layout CSS contract
