Files
fusion/docs/memory-backend-integration.md
ischindl 8fcf4bdbaa feat: Stash memory backend — session capture, per-chat backfill, opt-in vector search (#3494)
## Summary

Adds the **Stash memory backend** (`memory.backendType=stash`) that
connects Fusion's agent memory to the
[Stash](https://github.com/Fergana-Labs/stash) product — *knowledge
bases for the agent era* ([product site:
joinstash.ai](https://joinstash.ai)). Fusion becomes a first-class Stash
client: complete chat sessions and finished tasks are captured into
Stash, memory is recalled during chat, and Stash sessions are kept in
sync with the dashboard (including deletes and archival).

**Product:** <https://github.com/Fergana-Labs/stash> ·
[joinstash.ai](https://joinstash.ai)

## What's included

### 1. Stash memory backend (RUFU-068 / RUFU-121)
- New `StashMemoryBackend` (`memory.backendType=stash`) with `stashUrl`
/ `stashApiKey` settings (global secrets-store `stash-api-key` +
per-project override).
- **Complete-chat-session capture** keyed by ChatSession id.
- Sessions are classified into **per-project folders** (get-or-create,
`external_key fusion-<projectId>`, 1h per-process cache) and
**soft-deleted with their chat** via `DELETE /api/chat/sessions/:id`.
- Per-conversation **memory-focus** read-time scoping (new
`0066_chat_session_memory_focus.sql` migration — sequence renumbered
0059→0060→0061→0065→0066 as origin/main claimed the lower numbers);
event metadata enriched with `project` / `project_name` / `chat_title`.
- Recall queries normalized to single-keyword / explicit-OR ASCII (≤100
chars); shared normalizer export reused by per-turn recall.

### 2. Per-task executor transcript capture (RUFU-122)
Finished or failed tasks upload their executor transcript
(`agent-log.jsonl`) to Stash as a task session.

### 3. Bulk archive Stash sync (RUFU-125)
Archived task-planner chats soft-delete their Stash sessions on bulk
archival (paged). The snapshot of doomed session ids is taken *before*
the local bulk delete, and the Stash sync runs fire-and-forget so a
Stash stall can never delay local archival.

### 4. Per-chat "Preserve to Stash" backfill (RUFU-136)
A per-chat action that backfills a chat's full history into Stash, with
client-side idempotency and a pre-check that skips already-uploaded
content (fail-closed, no duplicate upload on transport failure).
- **Session-folder naming fix:** the first project folder is now named
"Fusion — &lt;project name&gt;" instead of the bare "Fusion" fallback
(the backfill now resolves the central-registry project name,
best-effort, never blocking the upload).

### 5. Opt-in semantic (vector) recall (RUFU-126)
`stashVectorSearch` setting (default `false` — **zero behavior change
until enabled**). For multi-word queries the backend tries Stash's
semantic-search endpoint first, then falls back byte-identically to the
keyword path. Definitive 404/405/501/503 responses are negatively cached
per process. Requires a patched Stash server (new endpoint +
`sentence-transformers` + embedding backfill); unpatched servers are
transparently bypassed after the first 404.

## Safety
- **Opt-in / inert by default:** the default backend remains `qmd`; the
Stash backend is inert until `memoryBackendType=stash` + `stashUrl` are
set.
- All Stash I/O is **best-effort, fail-closed, and non-blocking** — a
Stash outage never blocks chat, task completion, or archival. No
run-audit content is emitted.

## Testing
- Backfill + delete-sync suites (20/20), Stash backend suite (68/68),
executor memory / session capture suites, `memory-focus-recalling`,
description-guard — all green.
- `tsc` clean across core / engine / dashboard.
- Live verification: bulk backfill of 21/24 chats completed; the
"Preserve to Stash" action is idempotent on re-run.

## Changesets
- `@runfusion/fusion` **minor** — Stash memory backend + capture
(RUFU-068/121), per-task transcript (RUFU-122), bulk archive sync
(RUFU-125), per-chat backfill (RUFU-136), opt-in vector search
(RUFU-126)
- `@runfusion/fusion` **patch** — backfill session-folder naming fix


## Rebase Note (2026-08-23)

Rebased onto `origin/main` `3f448f7292` (v0.77.0-beta.7). Conflicts
resolved additively:
- `packages/core/src/postgres/schema-applier.ts` + test — upstream's
0062-0065 migrations (task/subtask splitting removal, AI merge review
reconciliation, task repository scope, FN-149 review convergence)
unioned with this PR's `chat_sessions.memory_focus` migration, which is
**renumbered 0065 → 0066** (upstream's FN-149 shipped 0065 canonically
on origin/main); `SCHEMA_BASELINE_VERSION` advances to `0066`.
- `packages/dashboard/app/components/ChatView.tsx` — upstream's docked
chat sidebar resize handlers unioned with the RUFU-136 "Preserve to
Stash" backfill handler.
- New commit: `settings.memory.*` stash-backend i18n keys added to all 6
secondary locales (RUFU-121/122 parity fix; `pnpm i18n:status` no longer
reports any violation introduced by this PR).

**Deploy note (operator environments that already ran a pre-rebase build
of this PR):** the memory-focus SQL may already be in the schema under
ledger row `0065`. Remap that row to `0066` (`UPDATE
fusion_schema_migrations SET version = '0066' WHERE version = '0065';`)
*before* first boot of a 0066-ceiling binary — otherwise the fresh
upstream `0065_fn_149_review_convergence_stage.sql` would be skipped as
"already applied". Clean databases (no prior memory-focus row) need no
action.

**CI note — Lint (lifecycle-column census) is red on the merge base:**
`pnpm check:lifecycle-columns --strict` fails identically on pure
`origin/main` `3f448f7292` with
`packages/core/src/db/legacy-adoption.ts: 0 -> 3` (3 column guards in
the U9b legacy-adoption table without a baseline entry or
`DELIBERATE-LITERAL` marker). Verified by running the census on a clean
origin/main checkout — inherited from the base, not introduced by this
PR. Fix belongs upstream; tracked separately.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added Stash memory integration with project configuration and optional
semantic search.
  * Added per-chat memory focus controls and a `/focus` command.
  * Added “Preserve to Stash” for uploading complete chat history.
  * Added automatic chat, task transcript, and completion-event capture.
* Added project-specific Stash session folders and archive/delete
synchronization.
* **Bug Fixes**
* Improved Stash folder naming and handling of missing branches during
no-commit tasks.
* **Documentation**
* Added setup, configuration, integration, vector-search, and
performance guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Fusion <noreply@runfusion.ai>
Co-authored-by: gsxdsm <gsxdsm@users.noreply.github.com>
2026-08-23 16:46:14 -07:00

6.3 KiB
Raw Blame History

Stash Memory Backend Integration

Fusion persists AI "memory" — task-completion shots, chat-session transcripts, recall hits, and per-conversation read focus — to a pluggable memory backend. The only memory backend currently wired into Fusion is Stash, a session-oriented event store. This document is the canonical operator guide for how Fusion talks to Stash, how it authenticates, how it isolates projects, and how conversation memory is captured and focused.

There is intentionally no TencentDB backend in this build. The backend enum only ever resolves stash (or the default qmd no-op); a TencentDB backend type, URL setting, or integration doc does not exist.


1. The Stash server

Stash exposes a small HTTP REST API used by Fusion for both read and write of memory events. All calls are scoped to the authenticated operator's own user id.

  • SEARCH (read): GET /api/v1/me/sessions/events/search?q=<query>&limit=<n>
  • CAPTURE (write): POST /api/v1/me/sessions/events/batch

The events sent to /events/batch must conform to a Stash event shape that requires a top-level event_type, agent_name, and session_id per event; a missing required field is rejected with HTTP 422. Fusion sets agent_name (default "fusion") and session_id on every event it uploads, so transcripts render correctly in the Stash /sessions/<sessionId> GUI.

2. Server URL and configuration

  • Setting key: memory.backendType — the resolved backend type. Only "stash" triggers Stash capture/recall. Any other value (including the default "qmd") is a no-op for both capture and read, and a non-Stash backend never reads secrets.
  • Setting key: memory.stashUrl — the Stash server base URL. When empty, Fusion falls back to the default http://127.0.0.1:3457 (a locally-hosted Stash daemon). The stored value is trimmed of trailing slashes.
  • Setting key: memory.stashApiKey — an optional per-project override for the API key. It is never committed to source; it is a runtime setting an operator can provide when they do not want to (or cannot) use the global secrets store.

3. Authentication

Fusion authenticates to the Stash server with an API key resolved in this precedence order:

  1. Per-project override memory.stashApiKey (settings) — wins if set.
  2. Global secrets store key stash-api-key (scope global) — read via the secrets-store revealSecret. This is the recommended mechanism; the key lives outside the repo in the operator's secret store and is never committed.

The API key is never hardcoded in Fusion source. Resolution degrades fail-closed: a missing or undecryptable secret resolves to an empty key (an unauthenticated request), so capture becomes a no-op rather than an error. Only a Stash backend ever reads secrets — a non-Stash or memory-disabled project triggers no secret read at all.

4. Per-project isolation

Memory events carry a provenance discriminator derived from the project root (e.g. fusion:<slug>). The discriminator is not the isolation mechanism:

Stash enforces isolation itself, scoping all reads and writes to the operator's own owner user id (owner_user_id IN accessible_scope_ids_sql(1)). The Fusion discriminator tag is provenance / grep-ability only — it lets an operator query "which Fusion project wrote this event" — and it is never confused with a required request field.

Because Stash scopes by operator identity, two Fusion projects belonging to different operators are naturally isolated, while a single operator's projects share the owner scope and are distinguishable by the discriminator tag.

5. Per-conversation memory focus (read-time scoping)

Fusion implements conversation focus so a recall hit is scoped to the conversation that produced it. The focus is persisted per chat session via the schema migration 0059_chat_session_memory_focus.sql (SCHEMA_BASELINE_VERSION = 0059), which adds a memory_focus column to the chat-session table.

At read time, the memory topic / focus (the text an operator or model optimizes a conversation around) is applied as a scoping filter on recall, so search within a focused conversation does not surface unrelated project memory. The focus value is carried through the memory read path and used as a Stash search topic parameter. This is a read-time scoping behavior — it does not rewrite what is captured, only what a focused conversation recalls.

Note: 0049_chat_session_memory_focus.sql is a clean-rebase-only artifact name and does not exist on this target. Origin's 0049 remains 0049_fn_8864_agent_activity_events.sql. The memory-focus migration here is the new 0059_*.sql, and no 0048_*.sql–0058_*.sql migration was deleted or modified.

6. Complete-chat-session capture

Fusion captures complete chat sessions into Stash (not merely per-task completion shots). A chat-store subscription turns the live conversation stream into per-message Stash memory events:

  • Per-message events — as each chat message is added, Fusion maps it to a capture event (user_message / assistant_message / tool_use, with agent_name, content, and tool_name for tool events) and progressively appends it to Stash. session_id is the Fusion ChatSession id, so the Stash /sessions/<sessionId> screen shows the full transcript.
  • Conversation-close flush — when a session transitions to a final status (archived), any remaining buffered-but-not-yet-appended messages are flushed as a final batch. The final flush is idempotent / dedup-safe: already-appended per-message events are never re-emitted.

Capture is best-effort / fail-closed / non-blocking:

  • A capture or secret-resolution failure never blocks or fails a chat or task completion.
  • A disabled memory backend or a non-Stash backend makes capture a no-op.
  • Captured content is written to the memory backend only — never to run-audit (run-audit rows carry ids/counts/outcomes only, per the FN-7158 rule).

Per-task completion capture

In addition to chat transcripts, Fusion emits a task_completion memory event (session id fusion-task-<taskId>) when a task completes, so a task's finishing state is recorded in the operator's memory. Like chat capture, this is completion-gated (at most once per task), best-effort, and non-blocking.