diff --git a/.changeset/fn-7544-artifact-cross-instance-live-refresh.md b/.changeset/fn-7544-artifact-cross-instance-live-refresh.md new file mode 100644 index 0000000000..f890cc0447 --- /dev/null +++ b/.changeset/fn-7544-artifact-cross-instance-live-refresh.md @@ -0,0 +1,7 @@ +--- +"@runfusion/fusion": patch +--- + +summary: Fix agent-created artifacts not appearing live in the dashboard artifacts view. +category: fix +dev: Root cause was cross-instance artifact-registration replication, not the route/hook/render path (all already correct). `TaskStore.registerArtifact()` never bumped `lastModified`, and `checkForChanges()` (the polling replicator that lets a second TaskStore instance on the same project — e.g. the dashboard's cached store vs. the engine's own store — mirror events it did not write itself) only ever diffed the `tasks` table, never `artifacts`. A store instance that did not perform the write could therefore never observe or re-emit `artifact:registered`, leaving an already-open Documents/task Artifacts gallery stale until a full reload. Fixed by bumping `lastModified` on artifact writes and adding a strictly-increasing `rowid`-cursor poll over the `artifacts` table in `checkForChanges()`. See `packages/core/src/__tests__/artifacts.test.ts` and `packages/dashboard/src/routes/__tests__/artifacts-route-integration.test.ts` for regression coverage. diff --git a/docs/storage.md b/docs/storage.md index 6169caf440..2588287013 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -73,6 +73,7 @@ - Task-linked artifact registration requires an active, non-archived task. Archived tasks are read-only for artifact writes; soft-deleted or missing tasks are rejected. - Retention follows the existing task lifecycle rather than a separate artifact policy: soft-deleted parent tasks keep artifact rows/files for forensics but normal live-reader APIs hide them; hard deletion from the active `tasks` table cascades artifact metadata through the `taskId` foreign key, and archive cleanup removes the task directory that contains task-scoped artifact binaries. Task-less artifacts live under `/.fusion/artifacts/` and are not tied to task archival cleanup. - Worktree DB hydration copies task-scoped artifact metadata for the current task/dependency graph alongside task rows and `task_documents`. It intentionally does not copy binary payload files, and it intentionally excludes task-less registry artifacts because dependency hydration is scoped to the active task graph. +- **Cross-instance live refresh (FN-7544).** A project can have more than one `TaskStore` instance open against the same DB at once (e.g. the dashboard's cached `getOrCreateProjectStore` instance vs. the engine's own internally-constructed store, or two dashboard processes). `registerArtifact()` calls `bumpLastModified()` and `checkForChanges()`'s 1s polling loop diffs the `artifacts` table by a strictly-increasing `rowid` cursor (not a timestamp, to avoid millisecond ties), re-emitting `artifact:registered` on any instance that did not perform the write itself. Without this, an already-open Documents/task Artifacts gallery served by a different store instance than the one an agent wrote through would never receive the live event and would show a stale list until a full reload re-ran the initial fetch. Agent-facing registration tools are documented in [Artifact registry tools](./agents.md#artifact-registry-tools), and the dashboard browsing surface is documented in [Artifacts View](./dashboard-guide.md#artifacts-view). diff --git a/packages/core/src/__tests__/artifacts.test.ts b/packages/core/src/__tests__/artifacts.test.ts index 1b7e2e18c4..30444cf90e 100644 --- a/packages/core/src/__tests__/artifacts.test.ts +++ b/packages/core/src/__tests__/artifacts.test.ts @@ -92,6 +92,91 @@ describe("TaskStore artifacts", () => { expect(registered).toHaveBeenCalledWith(artifact); }); + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * Reproduces FN-7544: an agent registers an artifact through one TaskStore instance (e.g. the + * engine's own store) while a SECOND TaskStore instance on the same project (e.g. the dashboard's + * cached getOrCreateProjectStore instance, or a second dashboard process) is the one whose SSE + * listeners actually serve the Documents/task Artifacts galleries. Before the fix, registerArtifact() + * never bumped lastModified and checkForChanges() never looked at the artifacts table at all, so the + * observer instance's `artifact:registered` subscribers never fired for artifacts written elsewhere — + * the exact symptom reported: the artifact exists in the DB (a plain GET/listArtifacts finds it) but + * nothing tells an already-open dashboard gallery to refresh, and any subscriber relying solely on the + * live event has no way to discover the row. + */ + it("a second TaskStore polling the same DB observes artifact:registered for artifacts it did not write", async () => { + const task = await store.createTask({ title: "Cross-instance artifact task", description: "Cross-instance artifact registration" }); + + const observer = new TaskStore(rootDir, join(rootDir, ".fusion-global-settings")); + await observer.init(); + await observer.watch(); + const observerRegistered = vi.fn(); + observer.on("artifact:registered", observerRegistered); + + try { + const artifact = await store.registerArtifact({ + type: "document", + title: "Written by another instance", + content: "# Cross-instance", + authorId: "agent-cross-instance", + authorType: "agent", + taskId: task.id, + }); + + // Drive the observer's poll cycle directly so we don't wait on the 1s interval. + await (observer as unknown as { checkForChanges: () => Promise }).checkForChanges(); + + expect(observerRegistered).toHaveBeenCalledTimes(1); + expect(observerRegistered).toHaveBeenCalledWith(expect.objectContaining({ id: artifact.id, title: "Written by another instance" })); + + // The observer must not re-emit on a second poll cycle for the same row. + observerRegistered.mockClear(); + await (observer as unknown as { checkForChanges: () => Promise }).checkForChanges(); + expect(observerRegistered).not.toHaveBeenCalled(); + + // Confirm the artifact is also readable via the observer's own listArtifacts query + // (the store-query surface, independent of the live SSE event). + const observerList = await observer.listArtifacts({ taskId: task.id }); + expect(observerList.map((a) => a.id)).toContain(artifact.id); + } finally { + await observer.close(); + } + }); + + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * FN-7544: the cross-instance polling fix must never leak an artifact registered in one project's + * TaskStore/DB into a completely separate project's store or its `artifact:registered` subscribers. + * Each project is a distinct rootDir/DB file, so this proves the fix stays scoped per-project. + */ + it("does not leak artifact:registered or listArtifacts rows across separate project stores", async () => { + const otherRootDir = makeTmpDir(); + const otherStore = new TaskStore(otherRootDir, join(otherRootDir, ".fusion-global-settings")); + await otherStore.init(); + await otherStore.watch(); + const otherRegistered = vi.fn(); + otherStore.on("artifact:registered", otherRegistered); + + try { + await store.registerArtifact({ + type: "document", + title: "Project A only", + content: "# Project A", + authorId: "agent-project-a", + authorType: "agent", + }); + + await (otherStore as unknown as { checkForChanges: () => Promise }).checkForChanges(); + + expect(otherRegistered).not.toHaveBeenCalled(); + const otherList = await otherStore.listArtifacts(); + expect(otherList).toHaveLength(0); + } finally { + await otherStore.close(); + await rm(otherRootDir, { recursive: true, force: true }); + } + }); + it("stores binary artifacts on disk under the task artifacts directory", async () => { const task = await store.createTask({ description: "Binary artifact task" }); const data = Buffer.from([0, 1, 2, 3, 255]); diff --git a/packages/core/src/store.ts b/packages/core/src/store.ts index 8d69a5e26c..03bf74d46d 100644 --- a/packages/core/src/store.ts +++ b/packages/core/src/store.ts @@ -1664,6 +1664,19 @@ export class TaskStore extends EventEmitter { private lastKnownModified: number = 0; /** ISO timestamp of last poll — used to filter changed tasks */ private lastPollTime: string | null = null; + /** + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * Cross-instance artifact registration (e.g. an engine-owned TaskStore writing while the dashboard + * reads through a separately-cached `getOrCreateProjectStore` instance) was invisible to the polling + * change-detector: registerArtifact() never bumped lastModified and checkForChanges() only diffed the + * `tasks` table, so a second TaskStore instance on the same project never observed or re-emitted + * `artifact:registered` for rows it did not write itself. Track the highest SQLite `rowid` this + * instance has already accounted for (seeded at watch() startup, advanced in-process on local writes + * and by the poll below) rather than a timestamp — `artifacts.createdAt` is millisecond-precision and + * ties with the poll's own "as of" timestamp are possible, while `rowid` is a strictly-increasing, + * never-reused integer so a `> cursor` comparison can never miss or double-count a row. + */ + private lastArtifactRowId = 0; /** One-shot startup sweep flag for clearing stale pause fields on done tasks. */ private donePauseBackfillDone = false; /** Short-lived startup memo for repeated slim listTasks reads before steady-state watch/polling. */ @@ -12466,6 +12479,14 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS} this.taskCache.set(task.id, { ...task }); } + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * Seed the artifact rowid cursor with the highest rowid already on disk so checkForChanges() only + * ever treats rows written by ANOTHER TaskStore instance (after this watch() call) as newly discovered. + */ + const artifactRowIdSeed = this.db.prepare("SELECT COALESCE(MAX(rowid), 0) as maxRowId FROM artifacts").get() as { maxRowId: number }; + this.lastArtifactRowId = artifactRowIdSeed.maxRowId; + try { await this.markLegacyAutoMergeStampsOnce(); } catch (err) { @@ -12664,6 +12685,25 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS} } } + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * Mirror the task-change replication above for the artifacts table: pick up rows inserted by + * ANOTHER TaskStore instance on this project since the last poll (rowid > lastArtifactRowId) and + * re-emit `artifact:registered` so already-open Documents/task Artifacts galleries in this process + * live-refresh. registerArtifact() advances lastArtifactRowId for this instance's own writes so + * this poll never double-emits for a row it already emitted directly. + */ + const changedArtifactRows = this.db + .prepare("SELECT rowid as _rowid, * FROM artifacts WHERE rowid > ? ORDER BY rowid ASC") + .all(this.lastArtifactRowId) as unknown as Array; + + for (const artifactRow of changedArtifactRows) { + if (artifactRow._rowid > this.lastArtifactRowId) { + this.lastArtifactRowId = artifactRow._rowid; + } + this.emit("artifact:registered", this.rowToArtifact(artifactRow)); + } + const elapsed = Date.now() - startTime; if (elapsed > 750) { storeLog.warn("checkForChanges took longer than expected", { @@ -13434,7 +13474,7 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS} } private insertArtifactRow(input: ArtifactCreateInput, id: string, now: string, stored: { uri?: string; sizeBytes?: number }): Artifact { - this.db.prepare( + const info = this.db.prepare( `INSERT INTO artifacts ( id, type, title, description, mimeType, sizeBytes, uri, content, authorId, authorType, taskId, metadata, createdAt, updatedAt ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, @@ -13455,6 +13495,17 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS} now, ); + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * Advance this instance's own artifact-poll rowid cursor past the row it just inserted so its next + * checkForChanges() cycle does not re-emit `artifact:registered` for a write it already emitted + * directly in registerArtifact() below. + */ + const insertedRowId = Number(info.lastInsertRowid); + if (insertedRowId > this.lastArtifactRowId) { + this.lastArtifactRowId = insertedRowId; + } + const row = this.db.prepare("SELECT * FROM artifacts WHERE id = ?").get(id) as ArtifactRow | undefined; if (!row) { throw new Error(`Failed to register artifact ${id}`); @@ -13501,6 +13552,16 @@ ${TASK_UPSERT_SQL_ASSIGNMENTS} }; const artifact = input.taskId ? await this.withTaskLock(input.taskId, register) : await register(); + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * bumpLastModified() is the signal checkForChanges() gates on before doing any polling work at all; + * without it, a second TaskStore instance on this project (dashboard vs engine, or two dashboard + * processes) never even looks at the artifacts table, so its SSE listeners silently never fire + * `artifact:registered` for artifacts written by the OTHER instance. insertArtifactRow() already + * advanced lastArtifactRowId past this row so this same instance's own poll cycle does not + * double-emit for the write we already emit directly below. + */ + this.db.bumpLastModified(); this.emit("artifact:registered", artifact); return artifact; } diff --git a/packages/dashboard/src/routes/__tests__/artifacts-route-integration.test.ts b/packages/dashboard/src/routes/__tests__/artifacts-route-integration.test.ts index 55adbf24be..af501504b1 100644 --- a/packages/dashboard/src/routes/__tests__/artifacts-route-integration.test.ts +++ b/packages/dashboard/src/routes/__tests__/artifacts-route-integration.test.ts @@ -1,6 +1,6 @@ // @vitest-environment node -import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import express from "express"; import { mkdtempSync, rmSync } from "node:fs"; import http from "node:http"; @@ -164,4 +164,122 @@ describe("artifacts route integration", () => { expect(res.status).toBe(200); expect(res.body).toEqual([]); }); + + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * FN-7544 surface enumeration: a task-less agent-authored registry artifact (no taskId) must appear + * in the global GET /api/artifacts listing exactly like task-scoped ones. + */ + it("a task-less agent-authored artifact appears in the global artifacts listing", async () => { + const artifact = await store.registerArtifact({ + type: "document", + title: "Registry-only note", + content: "# Task-less artifact", + authorId: "agent-registry", + authorType: "agent", + }); + + const res = await REQUEST(app, "GET", "/api/artifacts"); + + expect(res.status).toBe(200); + const body = res.body as ArtifactWithTask[]; + expect(body.map((a) => a.id)).toContain(artifact.id); + const found = body.find((a) => a.id === artifact.id); + expect(found?.authorType).toBe("agent"); + expect(found?.taskId).toBeFalsy(); + }); + + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * FN-7544 surface enumeration: GET /api/artifacts?taskId= must scope strictly to the requested task — + * artifacts from a different task or task-less registry rows must not leak in (project/task-scope + * isolation), while multiple artifacts for the SAME task must all be returned. + */ + it("?taskId= scopes strictly to the requested task and returns all of its artifacts", async () => { + const { task: taskA, artifact: artifactA } = await createTaskImageArtifact(); + const taskB = await store.createTask({ title: "Other task", description: "A different task" }); + const artifactB = await store.registerArtifact({ + type: "document", + title: "Other task note", + content: "# Belongs to task B", + authorId: "agent-7125", + authorType: "agent", + taskId: taskB.id, + }); + const registryArtifact = await store.registerArtifact({ + type: "document", + title: "Task-less note", + content: "# No task", + authorId: "agent-7125", + authorType: "agent", + }); + const artifactA2 = await store.registerArtifact({ + type: "document", + title: "Second note for task A", + content: "# Also task A", + authorId: "agent-7125", + authorType: "agent", + taskId: taskA.id, + }); + + const res = await REQUEST(app, "GET", `/api/artifacts?taskId=${encodeURIComponent(taskA.id)}`); + + expect(res.status).toBe(200); + const ids = (res.body as ArtifactWithTask[]).map((a) => a.id).sort(); + expect(ids).toEqual([artifactA.id, artifactA2.id].sort()); + expect(ids).not.toContain(artifactB.id); + expect(ids).not.toContain(registryArtifact.id); + }); + + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * FN-7544: a SECOND TaskStore instance against the same DB (mirroring the dashboard-vs-engine or + * two-process scenario) must observe artifact:registered for a write it did not perform once its poll + * cycle runs, and must serve the row through its own listArtifacts/GET /api/artifacts — not just the + * originating instance. This is the store-level fix under test, exercised through the HTTP route. + */ + it("a live-registered artifact is served by a second polling store instance's route", async () => { + /* + * FNXC:ArtifactRegistry 2026-07-04-20:10: + * The beforeEach `store` uses inMemoryDb:true, which cannot be shared across two TaskStore + * instances, so this scenario needs its own file-backed pair of stores against the same rootDir to + * reproduce two real processes polling the same on-disk DB. + */ + const crossRootDir = mkdtempSync(join(tmpdir(), "artifacts-route-cross-root-")); + const crossGlobalDir = mkdtempSync(join(tmpdir(), "artifacts-route-cross-global-")); + const writerStore = new TaskStore(crossRootDir, crossGlobalDir); + await writerStore.init(); + const observerStore = new TaskStore(crossRootDir, crossGlobalDir); + await observerStore.init(); + await observerStore.watch(); + const observerApp = express(); + observerApp.use(express.json()); + observerApp.use("/api", createApiRoutes(observerStore)); + + try { + const registered = vi.fn(); + observerStore.on("artifact:registered", registered); + + const artifact = await writerStore.registerArtifact({ + type: "document", + title: "Cross-instance route artifact", + content: "# Cross-instance", + authorId: "agent-cross-instance", + authorType: "agent", + }); + + await (observerStore as unknown as { checkForChanges: () => Promise }).checkForChanges(); + + expect(registered).toHaveBeenCalledTimes(1); + + const res = await REQUEST(observerApp, "GET", "/api/artifacts"); + expect(res.status).toBe(200); + expect((res.body as ArtifactWithTask[]).map((a) => a.id)).toContain(artifact.id); + } finally { + observerStore.close(); + writerStore.close(); + rmSync(crossRootDir, { recursive: true, force: true }); + rmSync(crossGlobalDir, { recursive: true, force: true }); + } + }); });