feat(FN-2523): add safe restart restore lifecycle diagnostics

- Add ProjectEngine restore lifecycle core to perform safe restarts and surface detailed restore state transitions
- Expose restore diagnostics through remote-access status types and settings/memory route context, including legacy API mapping updates
- Add comprehensive regression coverage for restore lifecycle behavior in engine and dashboard headless remote-access tests
- Document the restore lifecycle contract in architecture/settings docs and include a patch changeset for @runfusion/fusion
This commit is contained in:
Fusion
2026-04-26 05:18:12 -07:00
committed by gsxdsm
parent 2ce8b1c249
commit dd81a12e77
12 changed files with 956 additions and 19 deletions

View File

@@ -361,7 +361,11 @@ Implemented in `agent-heartbeat.ts`:
- Redaction is applied to command previews and emitted log lines before publishing status/log events.
- Deterministic stop semantics: graceful shutdown (`SIGTERM`) first, bounded wait, then force-kill fallback (`SIGKILL`).
- Safe provider switching is stop-first: active provider fully stops before target start is attempted; failed starts emit `switch_failed` terminal status.
- `ProjectEngine.start()` instantiates a per-project tunnel manager and exposes it through `getRemoteTunnelManager()` for API/UI consumers.
- `ProjectEngine.start()` instantiates a per-project tunnel manager and applies startup restore policy from `remoteAccess.lifecycle`:
- restore is attempted only when `rememberLastRunning` is true, a prior-running marker exists, provider config is valid, and runtime prerequisites are available.
- restore skips/failures are non-fatal to engine startup and clear stale running markers to avoid restart loops.
- Manual lifecycle remains explicit: only `startRemoteTunnel()` / `stopRemoteTunnel()` transitions mutate runtime state; provider/settings updates do not auto-start tunnels.
- `ProjectEngine` exposes restore diagnostics via `getRemoteTunnelRestoreDiagnostics()` (`applied|skipped|failed` + machine-readable reason).
### Multi-runtime support + IPC
- Runtime contracts: `project-runtime.ts`
@@ -388,6 +392,8 @@ Implemented in `agent-heartbeat.ts`:
Key server capabilities:
- REST APIs for tasks, git, GitHub, agents, missions, planning, automations/routines, settings
- Remote access APIs (`/api/remote/*`) for provider config, activation, tunnel lifecycle, status, token issuance, authenticated URL generation, and QR payload generation
- `/api/remote/tunnel/start` and `/api/remote/tunnel/stop` are the only lifecycle transition endpoints.
- `/api/remote/status` includes tunnel status plus restore diagnostics (`restore.outcome` + `restore.reason`) with parity between dashboard and headless `fn serve` runtimes.
- Remote auth handoff endpoints:
- `POST /api/remote-access/auth/login-url` (daemon-auth protected) issues a tokenized phone-login URL for either `persistent` or `short-lived` mode.
- `GET /remote-login?rt=<token>` (public) validates remote token strategy and redirects to dashboard auth handoff (`/?token=<daemonToken>` when daemon auth is enabled, otherwise `/`).

View File

@@ -198,9 +198,9 @@ The canonical persisted shape is a nested `remoteAccess` object.
| `remoteAccess.tokenStrategy.shortLived.enabled` | `boolean` | `false` | Enables short-lived token generation. |
| `remoteAccess.tokenStrategy.shortLived.ttlMs` | `number` | `900000` | Default short-lived token TTL in milliseconds (15 minutes). |
| `remoteAccess.tokenStrategy.shortLived.maxTtlMs` | `number` | `86400000` | Maximum allowed short-lived token TTL (24 hours). |
| `remoteAccess.lifecycle.rememberLastRunning` | `boolean` | `false` | Restore prior running tunnel at startup when valid. |
| `remoteAccess.lifecycle.wasRunningOnShutdown` | `boolean` | `false` | Internal state flag persisted on shutdown. |
| `remoteAccess.lifecycle.lastRunningProvider` | `"tailscale" \| "cloudflare" \| null` | `null` | Internal last-known running provider for restore decisions. |
| `remoteAccess.lifecycle.rememberLastRunning` | `boolean` | `false` | Enables safe startup restore attempts when prior-running markers + prerequisites are valid. |
| `remoteAccess.lifecycle.wasRunningOnShutdown` | `boolean` | `false` | Internal marker written by runtime lifecycle management; explicit manual stop clears this to prevent unintended restart restore. |
| `remoteAccess.lifecycle.lastRunningProvider` | `"tailscale" \| "cloudflare" \| null` | `null` | Internal provider marker used for startup restore gating; stale markers are cleared when restore is skipped/failed. |
Patch semantics for `PUT /api/settings`:
- `remoteAccess` patches are **deep-merged** so sibling branches are preserved.
@@ -248,6 +248,11 @@ Runtime provider config/credential contract (engine remote-access manager):
- Missing/invalid credential references fail fast with `invalid_config` status/error behavior.
- Secret-bearing values are redacted in command previews and emitted tunnel logs before they are published to subscribers.
Runtime lifecycle semantics:
- Provider/settings edits remain manual-only and do not auto-start tunnel processes.
- Startup restore is best-effort and non-fatal; failed/skipped restore attempts surface machine-readable diagnostics through `/api/remote/status` and do not loop indefinitely.
- Tunnel status payloads redact secret values (persistent/short-lived tokens and tokenized URLs are never returned raw from status diagnostics).
Short-lived token bounds are enforced server-side:
- Minimum TTL: `60_000` ms (60s)
- Maximum TTL: `86_400_000` ms (24h)