diff --git a/.changeset/artifact-pipeline-and-gallery.md b/.changeset/artifact-pipeline-and-gallery.md index 86653960e1..0c580e774b 100644 --- a/.changeset/artifact-pipeline-and-gallery.md +++ b/.changeset/artifact-pipeline-and-gallery.md @@ -2,6 +2,6 @@ "@runfusion/fusion": minor --- -summary: Agents now save screenshots/wireframes/mocks as artifacts, shown in a redesigned category gallery with doc editing. +summary: Agents save screenshots, videos, HTML mockups, and PDFs as artifacts, shown in a new category gallery with doc editing. category: feature -dev: fn_artifact_register gains a `path` payload source (file copied into managed storage, MIME inference, image signature validation) and is now always exposed to executor sessions (previously missing in ephemeral mode) with worktree-relative path resolution and executing-task default taskId; executor/planning prompts instruct agents to register visual deliverables; new `GET`/`PATCH /api/artifacts/:id` routes plus `TaskStore.updateArtifact` and the `artifact:updated` SSE event power in-place doc editing in the new ArtifactsGallery (Images/Docs/PDFs/Videos/Audio/Other sections with per-category viewers, mobile-responsive). +dev: fn_artifact_register gains a `path` payload source (file copied into managed storage, MIME inference, image/video/PDF signature validation) and is now always exposed to executor sessions (previously missing in ephemeral mode) with worktree-relative path resolution and executing-task default taskId; executor/planning prompts instruct agents to register visual/media deliverables (images, videos, HTML mockups, PDFs); the media route serves HTTP byte ranges for video/audio seeking; video attachments (100MB cap) bridge into the registry like images; HTML docs render as live sandboxed previews; new `GET`/`PATCH /api/artifacts/:id` routes plus `TaskStore.updateArtifact` and the `artifact:updated` SSE event power in-place doc editing in the new ArtifactsGallery (Images/Docs/PDFs/Videos/Audio/Other sections with per-category viewers, mobile-responsive). diff --git a/docs/agents.md b/docs/agents.md index 44e170a083..8be00acfba 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -44,7 +44,7 @@ fn chat [message…] [--once] [--non-interactive] [--poll-ms ] Artifact tools operate on the shared artifact registry, so artifacts are visible across agents and tasks when the caller has the artifact ID or can discover it through filters. -- `fn_artifact_register` registers a `document`, `image`, `video`, `audio`, or `other` artifact with `title`, optional `description`, optional `mimeType`, and exactly one payload source: inline text `content`, a local file `path` (preferred for media the agent saved to disk — screenshots, wireframes, mockups, recordings; the file is copied into managed artifact storage with MIME inference and image signature validation), base64 `dataBase64` image bytes, or a `uri` reference for media stored elsewhere. Executor sessions resolve relative `path` values against the task worktree and default `taskId` to the executing task; task-scoped heartbeat sessions default `taskId` to the assigned task; dashboard chat uses the `dashboard-chat` author and requires `task_id` because chat has no ambient task. +- `fn_artifact_register` registers a `document`, `image`, `video`, `audio`, or `other` artifact with `title`, optional `description`, optional `mimeType`, and exactly one payload source: inline text `content`, a local file `path` (preferred for media the agent saved to disk — screenshots, wireframes, mockups, screen recordings, PDF exports; the file is copied into managed artifact storage with MIME inference and image/video/PDF signature validation), base64 `dataBase64` image bytes, or a `uri` reference for media stored elsewhere. HTML mockups register as `type="document"` + `mimeType="text/html"` (inline `content` or `path`) and render as live sandboxed previews in the Artifacts view; PDFs (`mimeType="application/pdf"`, `path`) open in an embedded viewer; videos stream with range-request seeking. Executor sessions resolve relative `path` values against the task worktree and default `taskId` to the executing task; task-scoped heartbeat sessions default `taskId` to the assigned task; dashboard chat uses the `dashboard-chat` author and requires `task_id` because chat has no ambient task. - `fn_artifact_list` lists artifacts across agents and tasks with optional `type`, `authorId`, `taskId`, `search`, `limit`, and `offset` filters. Dashboard chat's scoped variant requires `task_id` and otherwise supports `type`, `authorId`, `search`, `limit`, and `offset` for that task. - `fn_artifact_view` fetches one artifact by `id`, returning registry metadata plus inline `content` when present or the stored `uri`/path reference for media artifacts. - Successful registration emits a best-effort `system` → `user` inbox notification to `DASHBOARD_USER_ID` with `artifactId`, `artifactType`, `title`, `authorId`, and optional `taskId` metadata. Notification delivery failures are logged and must never fail or roll back the artifact registration. diff --git a/docs/dashboard-guide.md b/docs/dashboard-guide.md index a6147ffe81..e63665a301 100644 --- a/docs/dashboard-guide.md +++ b/docs/dashboard-guide.md @@ -821,6 +821,8 @@ Features: - Use the tab-count badges to see the current counts for Project Files, Task Documents, and Artifacts; the Artifacts badge reflects the loaded `GET /api/artifacts` result set, including active search filters - Browse the category-driven gallery: artifacts are broken down into **Images**, **Docs**, **PDFs**, **Videos**, **Audio**, and **Other** content categories (PDFs are detected by MIME type/extension regardless of registry type). "All" renders one section per present category; the chip row filters to a single category, and chips only appear for categories that exist - Each category has a tailored experience: Images/Videos use a visual-first tile grid with hover metadata and a full-size lightbox; Docs open a full document viewer with rendered markdown; PDFs open an embedded viewer with an open-in-new-tab action; Audio renders inline player rows; Other renders compact download rows +- Video artifacts (agent-registered recordings, `path`-ingested MP4/WebM/MOV, and bridged video attachments) play with working seek because the media route serves HTTP byte ranges +- HTML doc artifacts (`mimeType: text/html`) render as **live sandboxed previews** by default in the document viewer (scripts allowed, same-origin denied), with a Preview/Source toggle and the same Edit mode as other docs - **Edit any inline-content doc in place**: the document viewer's **Edit** button switches to an editor whose **Save** persists through `PATCH /api/artifacts/:id` and live-refreshes open galleries via the `artifact:updated` registry event; binary-backed documents stay read-only with a media link - Dismiss any viewer with the close button, backdrop click, or Escape - Read artifact metadata on cards, rows, and viewer footers: title, optional description, author ID, timestamp, size, and linked task ID when present diff --git a/docs/screenshots/artifacts-gallery.png b/docs/screenshots/artifacts-gallery.png index 88bcbc8b92..9c72782021 100644 --- a/docs/screenshots/artifacts-gallery.png +++ b/docs/screenshots/artifacts-gallery.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:d56581556d5c8351034cc32dcf0b81a151720c4bc54d03fe9cdf30beb6c84041 -size 192458 +oid sha256:a3e5594d78abad19040108f444067126f069594fa171981a5558b00f611024ea +size 128061 diff --git a/docs/screenshots/artifacts-html-preview.png b/docs/screenshots/artifacts-html-preview.png new file mode 100644 index 0000000000..d18a348be2 --- /dev/null +++ b/docs/screenshots/artifacts-html-preview.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a7fbf7a7e9f1ded015fcc022337f1f97215199ce26a23c004bfa0a691a5dcb5 +size 95961 diff --git a/docs/screenshots/artifacts-video-lightbox.png b/docs/screenshots/artifacts-video-lightbox.png new file mode 100644 index 0000000000..f804e9c23b --- /dev/null +++ b/docs/screenshots/artifacts-video-lightbox.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:49fee3eee4a8eea0f7e55760717dd05f31fe6397197ee6192112fb27f8721a60 +size 135050 diff --git a/docs/storage.md b/docs/storage.md index 516dfb6bde..0279f73195 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -76,11 +76,12 @@ - `artifacts` is the first-class metadata registry for generated or uploaded task artifacts. Rows store ID, `type` (`document`, `image`, `video`, `audio`, or `other`), title/description, MIME type, size, author identity/type, optional task linkage, metadata JSON, textual `content`, a relative `uri`, and timestamps; binary bytes are not stored in SQLite. - `TaskStore.registerArtifact()` writes task-scoped binary payloads under `/.fusion/tasks/{ID}/artifacts/` and task-less registry payloads under `/.fusion/artifacts/`, then records a relative `artifacts/` URI in SQLite. If the DB insert fails after a binary write, the store removes the orphaned file before surfacing the error. -- Image task attachments (`image/png`, `image/jpeg`, `image/gif`, `image/webp`) are bridged into the artifact registry by `TaskStore.addAttachment()` as `image` rows with `metadata.source: "attachment"` and a relative `attachments/` URI. This keeps one copy of the bytes under `/.fusion/tasks/{ID}/attachments/` while making the image discoverable through artifact list APIs and the Documents/Task Artifacts galleries. Non-image attachments remain attachment-only. Deleting an attachment also deletes its bridged artifact row before removing the attachment file so `/api/artifacts/:id/media` does not point at a deleted attachment. +- Image task attachments (`image/png`, `image/jpeg`, `image/gif`, `image/webp`) and video task attachments (`video/mp4`, `video/webm`, `video/quicktime`; 100MB cap vs 5MB for other attachments) are bridged into the artifact registry by `TaskStore.addAttachment()` as `image`/`video` rows with `metadata.source: "attachment"` and a relative `attachments/` URI. This keeps one copy of the bytes under `/.fusion/tasks/{ID}/attachments/` while making the image discoverable through artifact list APIs and the Documents/Task Artifacts galleries. Non-image attachments remain attachment-only. Deleting an attachment also deletes its bridged artifact row before removing the attachment file so `/api/artifacts/:id/media` does not point at a deleted attachment. - Inline text/document artifacts may store `content` directly in SQLite and therefore have no media file. The dashboard media route streams `GET /api/artifacts/:id/media` from disk when `uri` is present, accepting task-scoped artifact URIs under `artifacts/` and bridged image-attachment URIs under `attachments/`, or returns inline `content` with the persisted MIME type when no `uri` exists. - `getArtifact(id)` returns metadata by ID, `getArtifacts(taskId)` returns active-task artifacts newest-first, and `listArtifacts(...)` is the cross-agent query path with type/author/task/search filters and pagination. List reads hide artifacts whose parent task is soft-deleted while preserving task-less artifacts. - `updateArtifact(id, { title?, description?, content? })` powers the dashboard Artifacts view's in-place doc editing (`GET`/`PATCH /api/artifacts/:id`). Content edits are only allowed on inline-content rows (no `uri`); binary-backed rows accept metadata edits only, archived-task artifacts stay read-only, and successful updates emit `artifact:updated` for live gallery refresh. -- `fn_artifact_register` accepts a local file `path` (in addition to inline `content`/`dataBase64`): the tool reads the file (50 MB cap), infers the MIME type from the extension when omitted, signature-validates image payloads (PNG/JPEG/GIF/WebP magic bytes, SVG text sniff), and persists the bytes through `registerArtifact()`'s managed storage path so the registry row keeps a servable URI after worktrees are cleaned up. Executor-lane registrations resolve relative paths against the task worktree and default `taskId` to the executing task. +- `fn_artifact_register` accepts a local file `path` (in addition to inline `content`/`dataBase64`): the tool reads the file (50 MB cap), infers the MIME type from the extension when omitted, signature-validates image payloads (PNG/JPEG/GIF/WebP magic bytes, SVG text sniff), video payloads (mp4/mov `ftyp` box, WebM EBML header), and PDF payloads (`%PDF-` prefix), and persists the bytes through `registerArtifact()`'s managed storage path so the registry row keeps a servable URI after worktrees are cleaned up. Executor-lane registrations resolve relative paths against the task worktree and default `taskId` to the executing task. HTML mockups register as `type="document"` + `mimeType="text/html"` (via `content` or `path`) and render as live sandboxed previews in the Artifacts view. +- `GET /api/artifacts/:id/media` serves HTTP byte ranges (`Accept-Ranges: bytes`, 206 + `Content-Range` for single ranges, 416 for unsatisfiable ranges) so `