feat(changelog): U6 — docs, agent guidance, and structured changeset
Update AGENTS.md, RELEASING.md, contributing.md with the structured changeset format guide. Add .changeset/README.md template. Add changeset for this change. Update distill-release-notes.mjs with final version.
This commit is contained in:
gsxdsm
37
.changeset/README.md
Normal file
37
.changeset/README.md
Normal file
@@ -0,0 +1,37 @@
|
||||
# Changeset Format Guide
|
||||
|
||||
Each changeset file in this directory describes one user-facing change for release notes.
|
||||
|
||||
## Required body format
|
||||
|
||||
```
|
||||
---
|
||||
"@runfusion/fusion": minor
|
||||
---
|
||||
|
||||
summary: Add a Command Center productivity control for LOC backfills.
|
||||
category: feature
|
||||
dev: Uses the new `fn_backfill_loc` tool; settings key `commandCenter.locBackfill`.
|
||||
```
|
||||
|
||||
## Fields
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `summary` | Yes | One line, user-facing, max 120 chars. Describe what changed for the operator. |
|
||||
| `category` | Yes | One of: `feature`, `fix`, `breaking`, `security`, `performance`, `internal`. |
|
||||
| `dev` | No | Developer or migration detail. Preserved in per-package CHANGELOGs but excluded from distilled release notes. |
|
||||
|
||||
## Audience
|
||||
|
||||
The `summary` is the only content that appears in end-user release notes by default. Write for Fusion operators — describe behavior, fixes, and what changed. Avoid internal class names, file paths, and implementation detail.
|
||||
|
||||
## Bump types
|
||||
|
||||
- `patch` — bug fixes, internal changes
|
||||
- `minor` — new features, CLI additions, tools
|
||||
- `major` — breaking changes
|
||||
|
||||
## Validation
|
||||
|
||||
Run `pnpm check:changesets` to validate. The linter runs in the PR-check gate and `test:gate`. Legacy freeform changesets pass with a warning during the transition period.
|
||||
7
.changeset/structured-changeset-format.md
Normal file
7
.changeset/structured-changeset-format.md
Normal file
@@ -0,0 +1,7 @@
|
||||
---
|
||||
"@runfusion/fusion": minor
|
||||
---
|
||||
|
||||
summary: Structured changeset format with AI-distilled release notes for cleaner, user-facing changelogs.
|
||||
category: feature
|
||||
dev: Changeset bodies now use labeled fields (summary, category, dev). A linter enforces the format in the PR gate. Release notes are distilled into grouped, end-user-facing sections. See .changeset/README.md for the format guide.
|
||||
21
AGENTS.md
21
AGENTS.md
@@ -44,6 +44,27 @@ Bump types:
|
||||
|
||||
Do **NOT** create changesets for AGENTS.md/README/internal docs, CI config, or behavior-preserving refactors. `@fusion/core`, `@fusion/dashboard`, and `@fusion/engine` are private.
|
||||
|
||||
#### Changeset body format (required)
|
||||
|
||||
Each changeset body must use labeled fields — not freeform paragraphs. The `summary` is the only content that appears in end-user release notes. The audience is Fusion operators, not developers reading internals.
|
||||
|
||||
```markdown
|
||||
---
|
||||
"@runfusion/fusion": minor
|
||||
---
|
||||
|
||||
summary: Add a Command Center productivity control for LOC backfills.
|
||||
category: feature
|
||||
dev: Uses the new `fn_backfill_loc` tool; settings key `commandCenter.locBackfill`.
|
||||
```
|
||||
|
||||
Fields:
|
||||
- `summary` (required) — one line, user-facing, max 120 chars. Describe what changed for the operator, not implementation detail.
|
||||
- `category` (required) — one of: `feature`, `fix`, `breaking`, `security`, `performance`, `internal`.
|
||||
- `dev` (optional) — developer/migration detail. Preserved in per-package CHANGELOGs but excluded from distilled release notes.
|
||||
|
||||
A linter (`pnpm check:changesets`) validates this format and runs in the PR-check gate. Legacy freeform changesets pass with a warning during the transition period; use `--strict` to fail on legacy format.
|
||||
|
||||
### Releasing
|
||||
|
||||
Use only:
|
||||
|
||||
20
RELEASING.md
20
RELEASING.md
@@ -20,7 +20,24 @@ This will prompt you to:
|
||||
- Choose the semver bump type (patch, minor, major)
|
||||
- Write a summary of the change
|
||||
|
||||
A markdown file will be created in the `.changeset/` directory. Commit this file along with your code changes.
|
||||
Then edit the created changeset file to use the structured body format:
|
||||
|
||||
```markdown
|
||||
---
|
||||
"@runfusion/fusion": minor
|
||||
---
|
||||
|
||||
summary: Add a Command Center productivity control for LOC backfills.
|
||||
category: feature
|
||||
dev: Uses the new `fn_backfill_loc` tool; settings key `commandCenter.locBackfill`.
|
||||
```
|
||||
|
||||
Fields:
|
||||
- `summary` (required) — one line, user-facing, max 120 chars.
|
||||
- `category` (required) — one of: `feature`, `fix`, `breaking`, `security`, `performance`, `internal`.
|
||||
- `dev` (optional) — developer/migration detail.
|
||||
|
||||
A markdown file will be created in the `.changeset/` directory. Commit this file along with your code changes. Validate with `pnpm check:changesets`.
|
||||
|
||||
### 2. Version PR is created automatically
|
||||
|
||||
@@ -29,6 +46,7 @@ When changesets are merged to `main`, the `version.yml` workflow automatically o
|
||||
- Consumes all pending changeset files
|
||||
- Bumps package versions according to the changeset declarations
|
||||
- Generates/updates `CHANGELOG.md` files for affected packages
|
||||
- Distills the version's changeset summaries into grouped, end-user-facing release notes in the root `CHANGELOG.md`
|
||||
|
||||
### 3. Merge the Version PR to release
|
||||
|
||||
|
||||
@@ -231,7 +231,7 @@ Use the default lane for normal local iteration before PRs. Run `test:deep` when
|
||||
Fusion uses Changesets + version PR workflow.
|
||||
|
||||
- See [RELEASING.md](../RELEASING.md) for release flow details.
|
||||
- For published package behavior changes, include a changeset.
|
||||
- For published package behavior changes, include a changeset using the structured body format (`summary`, `category`, optional `dev` fields). See the changeset format guide in [`.changeset/README.md`](../.changeset/README.md).
|
||||
|
||||
## Code Signing
|
||||
|
||||
|
||||
377
docs/plans/2026-06-23-003-feat-better-changelog-plan.md
Normal file
377
docs/plans/2026-06-23-003-feat-better-changelog-plan.md
Normal file
@@ -0,0 +1,377 @@
|
||||
---
|
||||
title: "Better Changelog — Structured Changesets + AI-Distilled Release Notes"
|
||||
date: 2026-06-23
|
||||
type: feat
|
||||
status: draft
|
||||
origin: user request
|
||||
deepened: false
|
||||
---
|
||||
|
||||
# Better Changelog — Structured Changesets + AI-Distilled Release Notes
|
||||
|
||||
## Summary
|
||||
|
||||
Replace the current dense, freeform changeset paragraphs with a **structured changeset body format** (category + end-user summary + optional dev detail) enforced by a linter, and add an **AI-powered release-notes distiller** that turns a release's collected changesets into clean, grouped, end-user-facing release notes. The distiller reuses the existing `createFnAgent` model-call seam, falls back to deterministic category-grouped rendering when no model is available, and feeds both the root `CHANGELOG.md` and GitHub Release notes through a single unified pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Fusion's changelog pipeline has two problems:
|
||||
|
||||
1. **Changeset content is unwieldy.** AI agents author dense, multi-paragraph technical changesets (some 800+ words of internal implementation detail) that aggregate into a 10,000+ line root `CHANGELOG.md`. The content is written for developers, not the Fusion operators who consume release notes.
|
||||
|
||||
2. **Release notes are inconsistent across paths.** The local release (`scripts/release.mjs`) extracts notes from the changeset-derived CHANGELOG via `extractVersionNotes`. The CI binary release (`.github/workflows/release.yml`) uses GitHub's `generate_release_notes: true`, which auto-generates from PR titles/commits — a completely different output. The dashboard's Update banner links to GitHub release notes, so users see whichever path produced the release.
|
||||
|
||||
The changeset *versioning engine* (cross-package fixed-group semver via `@changesets/cli`) is sound — the problem is content quality and output rendering, not versioning mechanics.
|
||||
|
||||
### Actors
|
||||
|
||||
- **AI agents** — primary changeset authors during task execution
|
||||
- **Fusion operators** — primary changelog/release-notes audience
|
||||
- **Release operator** — runs `pnpm release` locally or merges the CI version PR
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
- **R1.** Every changeset body follows a structured format: a category from a fixed set (`added`, `changed`, `fixed`, `deprecated`, `removed`, `security`) and a concise end-user-facing summary.
|
||||
- **R2.** A linter validates changeset bodies and runs in CI, rejecting non-conforming changesets before merge.
|
||||
- **R3.** A release-notes distiller consumes structured changesets and produces grouped, end-user-facing release notes organized by category (Keep a Changelog style).
|
||||
- **R4.** The distiller operates in two modes: deterministic (always available, category-grouped markdown) and AI-polished (when model access is available, reusing the existing `createFnAgent` seam).
|
||||
- **R5.** Both release paths (local `release.mjs` and CI `release.yml`) produce the same distilled release notes — one source of truth.
|
||||
- **R6.** The root `CHANGELOG.md` per-version entry is the distilled end-user release notes, not the raw per-package technical aggregate.
|
||||
- **R7.** Per-package `CHANGELOG.md` files render structured changesets as clean bullet points via a custom changelog function.
|
||||
|
||||
---
|
||||
|
||||
## Key Technical Decisions
|
||||
|
||||
### KTD1: Keep the changeset versioning engine, impose structured body schema
|
||||
|
||||
The `@changesets/cli` versioning engine correctly handles the fixed-group cross-package semver bumping defined in `.changeset/config.json`. Re-implementing that is real risk with no user-facing benefit. The problem is changeset *content quality*. We keep the engine and impose a structured body format enforced by a linter.
|
||||
|
||||
**Directional format** (implementation may refine the exact parsing rules):
|
||||
|
||||
```markdown
|
||||
---
|
||||
"@runfusion/fusion": minor
|
||||
---
|
||||
|
||||
added: Command Center productivity control for previewing and applying historical LOC backfills from the dashboard.
|
||||
```
|
||||
|
||||
- First non-empty line after frontmatter: `<category>: <end-user-facing summary>`
|
||||
- Category must be one of: `added`, `changed`, `fixed`, `deprecated`, `removed`, `security`
|
||||
- Optional subsequent paragraph: developer-facing detail (consumed by the distiller for AI context, omitted from end-user output by default)
|
||||
|
||||
### KTD2: Custom changeset changelog function
|
||||
|
||||
The changeset body renders into per-package `CHANGELOG.md` entries via the changelog function configured in `.changeset/config.json` (currently `"@changesets/cli/changelog"`). Without custom rendering, structured fields would appear raw (e.g., `added: some summary`). A custom changelog function parses the structured body and renders it as a clean bullet point. Developer detail is omitted from per-package entries (it lives in the changeset source file until consumed).
|
||||
|
||||
### KTD3: Two-mode distiller (deterministic + AI)
|
||||
|
||||
Release notes must work in both local and CI environments. The distiller's **deterministic mode** groups changesets by category and renders markdown — always available, no model call, no credentials. The **AI mode** passes structured entries to `createFnAgent` (with `tools: "readonly"`, mirroring `packages/dashboard/src/pr-metadata-generator.ts`) for a polished, grouped, end-user summary. AI mode activates when model access is available; deterministic mode is the fallback. Both modes consume the same structured changeset input, so the output shape is consistent regardless of mode.
|
||||
|
||||
### KTD4: Root CHANGELOG becomes distilled end-user notes
|
||||
|
||||
The root `CHANGELOG.md` is the user-facing artifact (linked from the dashboard Update banner via `packages/dashboard/app/components/UpdateAvailableBanner.tsx`). Each version's entry becomes the distilled, category-grouped end-user release notes. Per-package `CHANGELOG.md` files retain developer-facing technical detail (rendered cleanly via KTD2's custom function). Historical entries are not backfilled — only future releases get the new treatment.
|
||||
|
||||
### KTD5: CI model access is optional (deterministic fallback)
|
||||
|
||||
The distiller's AI mode needs model credentials. CI may not have these configured. Rather than making AI distillation a hard CI requirement, the distiller falls back to deterministic mode in CI (still a major improvement — clean category-grouped notes from structured summaries). When a CI model secret is configured, AI mode activates automatically. The local release path always uses AI mode (the operator's machine has model access).
|
||||
|
||||
---
|
||||
|
||||
## High-Level Technical Design
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["AI agent authors<br/>structured changeset"] --> B["Linter validates<br/>CI + local"]
|
||||
B -->|valid| C["Changeset committed<br/>to .changeset/"]
|
||||
B -->|invalid| A
|
||||
C --> D["Release triggered<br/>local release.mjs or CI version.yml"]
|
||||
D --> E["Distiller reads structured<br/>changesets BEFORE versioning"]
|
||||
D --> F["changesets/action<br/>version bump + per-package CHANGELOG<br/>(version.yml: npm; release.yml: GitHub Release)"]
|
||||
E --> G{Mode}
|
||||
G -->|AI available| H["createFnAgent<br/>polished end-user notes"]
|
||||
G -->|No model| I["Deterministic<br/>category-grouped notes"]
|
||||
H --> J["Distilled release notes"]
|
||||
I --> J
|
||||
F --> K["Per-package CHANGELOGs<br/>via custom changelog function"]
|
||||
J --> L["Root CHANGELOG<br/>end-user notes per version"]
|
||||
J --> M["GitHub Release body"]
|
||||
```
|
||||
|
||||
**Key sequencing constraint:** the distiller must read `.changeset/*.md` files *before* `changeset version` runs, because versioning consumes and deletes the changeset files. In `release.mjs`, the distillation call happens after authorization but before `pnpm release:version`. The distilled output is held in memory (or a temp file) and used after the CHANGELOG sync to (a) replace that version's root CHANGELOG entry and (b) feed the GitHub release `--notes-file`.
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
### In scope
|
||||
|
||||
- Structured changeset body format + parser + linter
|
||||
- Migration of existing pending `.changeset/*.md` files to the new format
|
||||
- Custom changeset changelog function for clean per-package rendering
|
||||
- Release-notes distiller (deterministic + AI modes)
|
||||
- Integration into both release paths (local `release.mjs` + CI `release.yml`)
|
||||
- Root `CHANGELOG.md` per-version entries become distilled end-user notes
|
||||
- Documentation updates (`AGENTS.md`, `RELEASING.md`, `docs/contributing.md`)
|
||||
|
||||
### Out of scope (non-goals)
|
||||
|
||||
- Re-implementing cross-package semver versioning (the changeset versioning engine stays)
|
||||
- Backfilling historical `CHANGELOG.md` entries to the new format
|
||||
- Changing the npm publishing mechanism (OIDC via `version.yml`)
|
||||
- Changing the binary build/signing pipeline (`release.yml` build legs)
|
||||
- Dashboard UI changes (the Update banner continues to link to GitHub releases)
|
||||
|
||||
### Deferred to follow-up work
|
||||
|
||||
- Changeset authoring automation (agent prompt updates to emit the structured format automatically — partially covered by AGENTS.md convention update)
|
||||
- Version-PR-preview release notes (showing distilled notes in the version PR body before merge)
|
||||
- Release-notes deduplication across packages in the fixed group
|
||||
- Migration of `extractVersionNotes` consumers if any remain after distiller integration
|
||||
|
||||
---
|
||||
|
||||
## Implementation Units
|
||||
|
||||
### U1. Structured Changeset Schema, Parser, and Linter
|
||||
|
||||
**Goal:** Define the structured changeset body format, implement a parser that extracts `{ category, userSummary, devDetail? }` from `.changeset/*.md` bodies, and a linter that validates all pending changesets against the schema.
|
||||
|
||||
**Requirements:** R1, R2
|
||||
|
||||
**Dependencies:** none
|
||||
|
||||
**Files:**
|
||||
- `scripts/lib/changeset-schema.mjs` (new) — parser + types
|
||||
- `scripts/check-changeset-format.mjs` (new) — linter entrypoint
|
||||
- `scripts/__tests__/changeset-schema.test.mjs` (new) — parser tests
|
||||
- `scripts/__tests__/check-changeset-format.test.mjs` (new) — linter tests
|
||||
|
||||
**Approach:**
|
||||
|
||||
The parser reads a changeset file in two phases: (1) parse the YAML frontmatter for package/bump-type (using the same lightweight parsing the existing release scripts already do), and (2) parse the body for the structured fields. The body format is: first non-empty line after frontmatter is `<category>: <user-facing summary>`, optional subsequent lines are developer detail. Categories are the Keep a Changelog set: `added`, `changed`, `fixed`, `deprecated`, `removed`, `security`.
|
||||
|
||||
The linter scans all `.changeset/*.md` files (excluding `config.json`), parses each, and reports violations: missing category prefix, invalid category, missing or empty summary, summary exceeding a character budget (directional: ~150 chars), or unparseable body. Exit code 1 on any violation. The linter is a standalone node script with no build dependency, so it can run in the lint CI job without needing compiled artifacts.
|
||||
|
||||
**Patterns to follow:** `scripts/check-no-nohup.mjs`, `scripts/check-no-kill-4040.mjs` — standalone validation scripts that read repo files and exit non-zero on violation. These already run as part of `test:gate`.
|
||||
|
||||
**Test scenarios:**
|
||||
- *Happy path:* parse a well-formed body (`added: some feature`) — returns `{ category: "added", userSummary: "some feature", devDetail: undefined }`
|
||||
- *Edge case:* body with category + summary + optional dev detail paragraph — devDetail populated
|
||||
- *Edge case:* body with extra blank lines between frontmatter and content — trimmed correctly
|
||||
- *Edge case:* summary at exactly the character budget boundary — accepted
|
||||
- *Error path:* body missing category prefix (e.g., `This adds a feature`) — linter rejects with clear message
|
||||
- *Error path:* invalid category (e.g., `enhanced: ...`) — linter lists valid categories
|
||||
- *Error path:* summary exceeding character budget — linter rejects
|
||||
- *Error path:* empty body after frontmatter — linter rejects
|
||||
- *Integration:* linter scans a directory of mixed valid/invalid `.changeset/*.md` files — reports all violations, exits 1
|
||||
|
||||
**Verification:** `node scripts/check-changeset-format.mjs` exits 0 when all changesets conform, exits 1 with per-file violation messages when any do not.
|
||||
|
||||
---
|
||||
|
||||
### U2. Migrate Existing Pending Changesets
|
||||
|
||||
**Goal:** Convert all current `.changeset/*.md` files to the new structured format so the linter passes on day one.
|
||||
|
||||
**Requirements:** R1
|
||||
|
||||
**Dependencies:** U1
|
||||
|
||||
**Files:**
|
||||
- All `.changeset/*.md` files (13 pending files as of this plan)
|
||||
|
||||
**Approach:**
|
||||
|
||||
Each existing changeset body is rewritten into the `category: summary` format. The category is inferred from the content (e.g., "Fix ..." → `fixed`, "Add ..." → `added`, "Breaking:" → `changed` or `removed`). The dense technical paragraph is compressed into a one-sentence end-user summary, with key technical context moved to the optional dev-detail paragraph. The frontmatter (package + bump type) is unchanged.
|
||||
|
||||
**Test expectation:** none — data migration. Verify all migrated files pass the linter from U1.
|
||||
|
||||
**Verification:** `node scripts/check-changeset-format.mjs` exits 0 against the migrated files.
|
||||
|
||||
---
|
||||
|
||||
### U3. Custom Changeset Changelog Function
|
||||
|
||||
**Goal:** Replace the default `@changesets/cli/changelog` with a custom function that parses structured changeset bodies and renders them as clean bullet points in per-package `CHANGELOG.md` files.
|
||||
|
||||
**Requirements:** R7
|
||||
|
||||
**Dependencies:** U1 (uses the parser)
|
||||
|
||||
**Files:**
|
||||
- `scripts/lib/changeset-changelog-function.mjs` (new) — custom changelog function
|
||||
- `.changeset/config.json` (modify) — point `changelog` field to the custom function
|
||||
- `scripts/__tests__/changeset-changelog-function.test.mjs` (new) — rendering tests
|
||||
|
||||
**Approach:**
|
||||
|
||||
The changesets library calls `getReleaseLine(changeset, type)` for each changeset when generating per-package CHANGELOG entries. The custom function parses the structured body (reusing U1's parser) and renders: `- **Added:** summary` (category title-cased and bolded). If the changeset body is not in structured format (e.g., a legacy entry that slipped through), it falls back to the raw body text so rendering never breaks. The `getDependencyReleaseLine` function passes through dependency bumps unchanged (these are mechanical and contain no user-facing content).
|
||||
|
||||
The `.changeset/config.json` `changelog` field changes from `"@changesets/cli/changelog"` to a path pointing at the custom function module.
|
||||
|
||||
**Patterns to follow:** The changesets changelog function interface (`getReleaseLine`, `getDependencyReleaseLine`). The fallback-to-raw pattern from `pr-metadata-generator.ts`'s `buildFallback`.
|
||||
|
||||
**Test scenarios:**
|
||||
- *Happy path:* structured body `added: feature text` → renders `- **Added:** feature text`
|
||||
- *Happy path:* each category title-cases correctly (`fixed` → `Fixed`, `deprecated` → `Deprecated`, etc.)
|
||||
- *Edge case:* body with dev detail → detail omitted from rendered line (per-package CHANGELOG is concise)
|
||||
- *Edge case:* legacy unstructured body → falls back to raw text rendering (no crash)
|
||||
- *Integration:* changesets/action calls `getReleaseLine` during `changeset version` → per-package CHANGELOG shows clean entries
|
||||
|
||||
**Verification:** Run `pnpm release:version --dry-run` (or equivalent) and inspect generated per-package CHANGELOG entries for clean rendering.
|
||||
|
||||
---
|
||||
|
||||
### U4. Release-Notes Distiller
|
||||
|
||||
**Goal:** Implement the distiller module with deterministic and AI modes that consume structured changesets and produce grouped, end-user-facing release notes.
|
||||
|
||||
**Requirements:** R3, R4
|
||||
|
||||
**Dependencies:** U1 (uses the parser)
|
||||
|
||||
**Files:**
|
||||
- `scripts/lib/release-notes-distiller.mjs` (new) — deterministic mode (pure JS, no engine imports)
|
||||
- `scripts/lib/release-notes-distiller-ai.ts` (new) — AI mode using `createFnAgent` from `@fusion/engine`
|
||||
- `scripts/__tests__/release-notes-distiller.test.mjs` (new) — deterministic mode tests
|
||||
|
||||
**Approach:**
|
||||
|
||||
**Deterministic mode** (`release-notes-distiller.mjs`): pure JS, no `@fusion/*` imports. Takes an array of parsed `StructuredChangeset` objects, groups by category in Keep a Changelog order (`Added`, `Changed`, `Deprecated`, `Removed`, `Fixed`, `Security`), and renders markdown:
|
||||
|
||||
```markdown
|
||||
## Added
|
||||
|
||||
- Command Center productivity control for LOC backfills
|
||||
- Editable global model pricing overrides
|
||||
|
||||
## Fixed
|
||||
|
||||
- Stop Planning Mode from auto-focusing on mobile
|
||||
- Fix stale durable agent task assignments
|
||||
```
|
||||
|
||||
Categories with no entries are omitted. Dev detail is excluded from the output.
|
||||
|
||||
**AI mode** (`release-notes-distiller-ai.ts`): a tsx-runnable module that imports `createFnAgent` from `@fusion/engine` (the same seam `packages/dashboard/src/pr-metadata-generator.ts` uses). It passes the structured changeset entries (category + summary, optionally enriched with dev detail) to the model with a system prompt that instructs it to produce a polished, grouped, end-user-facing summary. The model returns markdown in the same category-grouped shape, but with summaries rewritten for clarity and flow. The AI mode will default to the title-summarizer model setting (`resolveTitleSummarizerSettingsModel`) for consistency with other lightweight AI text tasks, pending the open question on whether a dedicated model setting is warranted.
|
||||
|
||||
**Fallback contract:** if the AI call fails, times out, or no model is configured, the distiller returns the deterministic output. The caller never sees an error from distillation — only degraded polish.
|
||||
|
||||
**Patterns to follow:** `packages/dashboard/src/pr-metadata-generator.ts` — the `createFnAgent({ tools: "readonly", onText, systemPrompt })` + accumulate + parse + fallback pattern.
|
||||
|
||||
**Test scenarios:**
|
||||
- *Happy path (deterministic):* 5 changesets across 3 categories → markdown with 3 category headings, entries grouped correctly, empty categories omitted
|
||||
- *Edge case (deterministic):* all changesets in one category → single heading section
|
||||
- *Edge case (deterministic):* empty changeset array → minimal/empty output
|
||||
- *Happy path (AI):* structured changesets → model produces grouped markdown (mocked `createFnAgent` in test)
|
||||
- *Error path (AI):* model timeout/error → falls back to deterministic output (no thrown error)
|
||||
- *Error path (AI):* no model configured → deterministic output returned directly
|
||||
- *Integration:* distiller output is valid markdown with category headings usable as GitHub release notes
|
||||
|
||||
**Verification:** Deterministic mode produces correct grouped markdown for representative changeset sets. AI mode falls back cleanly when the model is unavailable.
|
||||
|
||||
---
|
||||
|
||||
### U5. Integrate Distiller into Both Release Paths
|
||||
|
||||
**Goal:** Wire the distiller into local `release.mjs` and CI `release.yml` so both produce the same distilled release notes, and the root `CHANGELOG.md` entry per version is the distilled output.
|
||||
|
||||
**Requirements:** R5, R6
|
||||
|
||||
**Dependencies:** U3, U4
|
||||
|
||||
**Files:**
|
||||
- `scripts/release.mjs` (modify) — distill changesets before versioning, use distilled notes for root CHANGELOG + GitHub release
|
||||
- `scripts/lib/extract-version-notes.mjs` (modify or deprecate) — retained as fallback but superseded by distiller
|
||||
- `.github/workflows/release.yml` (modify) — use distilled notes instead of `generate_release_notes: true`
|
||||
- `.github/workflows/version.yml` (modify) — ensure root CHANGELOG sync uses distilled notes when versioning
|
||||
|
||||
**Approach:**
|
||||
|
||||
**Local path (`release.mjs`):** After authorization but before `pnpm release:version` (which consumes and deletes changeset files), read all `.changeset/*.md` files, parse them via U1's parser, and pass to U4's distiller (AI mode — the operator's machine has model access). Store the distilled notes. After `syncRootChangelog()` runs, replace that version's root CHANGELOG entry with the distilled notes. Use the distilled notes for the GitHub release `--notes-file` instead of `extractVersionNotes`.
|
||||
|
||||
**CI path (`release.yml`):** Replace `generate_release_notes: true` with `notes-file` pointing to a pre-distilled notes file. The notes file is produced during the version step (when changesets are consumed). Since CI may lack model credentials, the CI distiller runs in deterministic mode by default. When a CI model secret (e.g., `FUSION_CHANGELOG_MODEL_KEY`) is configured, AI mode activates. The `softprops/action-gh-release` action's `body` field receives the distilled notes.
|
||||
|
||||
**CI path (`version.yml`):** The `changesets/action` version step calls `pnpm release:version` which runs `changeset version`. The root CHANGELOG sync (currently only in local `release.mjs`'s `syncRootChangelog()`) needs to also run in CI so the version PR includes the distilled root CHANGELOG. Add a post-version step that runs the distiller in deterministic mode and patches the root CHANGELOG before the version PR is committed.
|
||||
|
||||
**Migration note:** `extractVersionNotes` is retained as a fallback for historical versions but is no longer the primary notes source. Existing tests in `scripts/__tests__/extract-version-notes.test.mjs` continue to pass.
|
||||
|
||||
**Test scenarios:**
|
||||
- *Happy path (local):* `release.mjs` distills changesets before versioning → GitHub release body matches distilled notes
|
||||
- *Happy path (local):* root CHANGELOG entry for the new version is the distilled end-user notes (not the raw per-package aggregate)
|
||||
- *Happy path (CI):* `release.yml` uses distilled notes file → GitHub release body matches distilled output
|
||||
- *Edge case (CI):* no model secret configured → deterministic distilled notes used (still grouped, clean)
|
||||
- *Edge case:* no changesets pending → release flow handles gracefully (no crash, minimal notes)
|
||||
- *Integration:* both paths produce the same notes shape for the same changeset set
|
||||
|
||||
**Verification:** A dry-run local release (`pnpm release --dry-run`) shows distilled notes. CI release workflow references a notes file rather than auto-generation.
|
||||
|
||||
---
|
||||
|
||||
### U6. Update Documentation and Conventions
|
||||
|
||||
**Goal:** Update all documentation that describes changeset authoring or the release flow to reflect the new structured format, linter, and distillation pipeline.
|
||||
|
||||
**Requirements:** R1, R2
|
||||
|
||||
**Dependencies:** U1, U5 (documents the final behavior)
|
||||
|
||||
**Files:**
|
||||
- `AGENTS.md` (modify) — update the "Finalizing Changes" changeset rules with the structured format, category list, and summary budget
|
||||
- `RELEASING.md` (modify) — update release flow to mention distillation
|
||||
- `docs/contributing.md` (modify) — update changeset convention section
|
||||
|
||||
**Approach:**
|
||||
|
||||
Update the AGENTS.md "Finalizing Changes" section to specify the new changeset body format: `category: user-facing summary` with the allowed category list, the character budget for summaries, and the optional dev-detail paragraph. Note that the linter enforces this in CI. Update the bump-type guidance (unchanged: patch/minor/major) but clarify the body format.
|
||||
|
||||
Update RELEASING.md to describe the distillation step in both release paths. Update contributing.md's changeset section to match.
|
||||
|
||||
Add FNXC comments to the new scripts documenting the date and requirement rationale.
|
||||
|
||||
**Test expectation:** none — documentation update.
|
||||
|
||||
**Verification:** Documentation accurately describes the new format and flow. No stale references to the old freeform changeset body convention.
|
||||
|
||||
---
|
||||
|
||||
## System-Wide Impact
|
||||
|
||||
**Affected parties:**
|
||||
- **AI agents** — every changeset authored during task execution must use the new structured format. The AGENTS.md update (U6) is the primary vector; agent prompt compliance is convention-driven.
|
||||
- **Release operator** — no change to the release command (`pnpm release`); distillation is automatic.
|
||||
- **Fusion operators** — release notes and root CHANGELOG become significantly more readable.
|
||||
- **CI** — `release.yml` and `version.yml` gain a distillation step; `pr-checks.yml` gains a changeset-format check.
|
||||
|
||||
**Affected surfaces:**
|
||||
- All future `.changeset/*.md` files (format change)
|
||||
- `.changeset/config.json` (changelog function change)
|
||||
- `scripts/release.mjs` (distiller integration)
|
||||
- Root `CHANGELOG.md` (per-version entries become distilled notes)
|
||||
- Per-package `CHANGELOG.md` files (rendered via custom function)
|
||||
- `.github/workflows/release.yml`, `.github/workflows/version.yml` (distillation integration)
|
||||
- `.github/workflows/pr-checks.yml` (linter in lint job)
|
||||
|
||||
---
|
||||
|
||||
## Risks & Dependencies
|
||||
|
||||
- **Risk: changeset format adoption by agents.** AI agents author changesets based on AGENTS.md conventions. If agents don't adopt the format, the linter blocks PRs. Mitigation: the AGENTS.md update (U6) is explicit, and the linter error messages list valid categories and the expected format.
|
||||
- **Risk: custom changelog function breaks changesets/action in CI.** The custom function replaces a well-tested default. Mitigation: fallback-to-raw rendering for non-conforming bodies (U3) ensures the function never crashes; test with `changeset version` locally before merging.
|
||||
- **Risk: distiller reads changesets at the wrong time.** The distiller must read `.changeset/*.md` before `changeset version` consumes them. Mitigation: U5 explicitly sequences the distillation call before `pnpm release:version` in `release.mjs`.
|
||||
- **Dependency: `createFnAgent` from `@fusion/engine`.** The AI distiller mode imports this. It requires model credentials and settings resolution. Mitigation: deterministic mode is the fallback; AI mode is opt-in via available credentials.
|
||||
- **Dependency: `softprops/action-gh-release` body vs notes-file.** The CI release needs to pass distilled notes. The action supports `body` (inline) or `body_path` (file). Verify which is cleaner for the CI integration.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **Linter enforcement level:** Should the changeset-format linter be part of the merge gate (`test:gate` / lint job in `pr-checks.yml`) from day one, or start as non-blocking in `full-suite.yml` and promote after a grace period? *Recommendation: start in the lint job (blocking) since U2 migrates all existing changesets in the same PR.*
|
||||
- **Summary character budget:** What is the right max length for the end-user summary? *Directional: 150 chars. Confirm during implementation.*
|
||||
- **Model setting for distiller AI mode:** Should the distiller use the existing `resolveTitleSummarizerSettingsModel` setting (shared with PR title summarization), or get a dedicated model setting? *Recommendation: reuse title-summarizer setting for now; add a dedicated setting only if quality or cost demands it.*
|
||||
285
docs/plans/2026-06-24-001-feat-better-changelog-plan.md
Normal file
285
docs/plans/2026-06-24-001-feat-better-changelog-plan.md
Normal file
@@ -0,0 +1,285 @@
|
||||
---
|
||||
title: "feat: Better changelog — structured changesets + AI-distilled release notes"
|
||||
type: feat
|
||||
date: 2026-06-24
|
||||
---
|
||||
|
||||
# feat: Better changelog — structured changesets + AI-distilled release notes
|
||||
|
||||
## Summary
|
||||
|
||||
Replace today's dense, agent-authored technical changeset paragraphs with a **structured, concise changeset schema** (end-user summary + category + optional dev detail), enforced by a linter. Add an **AI distillation step** at version time that transforms a release's collected changesets into clean, grouped, end-user-facing release notes, reusing the existing `createFnAgent` model-call seam. Unify both release paths (local `release.mjs` and CI `version.yml` / `release.yml`) behind a single distilled artifact so the root `CHANGELOG.md` and GitHub Release both carry the same user-facing notes.
|
||||
|
||||
## Problem Frame
|
||||
|
||||
AI agents currently author changesets as dense technical paragraphs — multi-sentence implementation detail, internal class names, and edge-case mechanics that serve developers, not the Fusion operators who read release notes. These aggregate into a 10,000+ line root `CHANGELOG.md` and flow unchanged into GitHub release notes via `extractVersionNotes`. The result is unwieldy: an end user reading "what changed in v0.46.0" wades through internal jargon about `reconcileOrphanedTaskDirs` recency windows and `dotGitPointerIsDangling` sentinels.
|
||||
|
||||
Two structural issues compound this:
|
||||
|
||||
1. No format constraint exists — changeset bodies are freeform markdown with no required fields, length cap, or audience guidance.
|
||||
2. The two release paths produce **different** notes: the local release (`release.mjs`) extracts from the changeset-derived `CHANGELOG.md`, while CI (`release.yml`) uses GitHub's `generate_release_notes: true` (auto-generated from PR titles/commits). Neither produces a curated, user-facing summary.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
### Changeset format
|
||||
|
||||
- R1. Each changeset body uses a structured schema with a required `summary` field (one line, user-facing, max 120 chars), a required `category` field (one of: `feature`, `fix`, `breaking`, `security`, `performance`, `internal`), and an optional `dev` field for developer/migration detail.
|
||||
- R2. The `summary` is the only content that flows into end-user release notes by default. The `dev` field is preserved in per-package CHANGELOGs but excluded from distilled release notes unless the distillation model judges it user-relevant.
|
||||
- R3. Existing freeform changesets are grandfathered during a transition period: the linter warns (not fails) when the structured fields are absent, giving the agent fleet time to adopt the new format.
|
||||
|
||||
### Linter
|
||||
|
||||
- R4. A changeset linter validates the structured schema and runs as part of the PR-check gate (`pr-checks.yml`) and `test:gate`, so malformed changesets block merge.
|
||||
- R5. The linter enforces `summary` length (max 120 chars), valid `category` enum, and that only `@runfusion/fusion` appears in the frontmatter bump declarations (matching the single-package-publish reality).
|
||||
|
||||
### Distillation
|
||||
|
||||
- R6. At version time, a distillation step reads the version's changesets, calls `createFnAgent` with a release-notes system prompt, and produces grouped, user-facing release notes organized by category (New, Fixed, Breaking, etc.).
|
||||
- R7. Distillation degrades gracefully: if the model call fails, times out, or returns unparseable output, the release proceeds using the structured `summary` lines as a fallback (bullet list by category), so a model outage never blocks a release.
|
||||
- R8. The distillation step respects the same settings-driven model resolution as other AI features (title-summarizer model settings), so operators can point it at any configured provider/model.
|
||||
|
||||
### Release integration
|
||||
|
||||
- R9. The local release (`release.mjs`) uses distilled notes for both the root `CHANGELOG.md` version section and the GitHub Release notes, replacing the current `extractVersionNotes` raw-aggregation path.
|
||||
- R10. The CI version workflow (`version.yml`) runs distillation after the changeset versioning step and writes the distilled notes into the root `CHANGELOG.md` before the version PR is created, so the merged version PR carries curated notes.
|
||||
- R11. The CI binary release workflow (`release.yml`) uses the root `CHANGELOG.md` version section (already distilled) for the GitHub Release body instead of `generate_release_notes: true`, so both paths produce identical curated notes.
|
||||
|
||||
### Root CHANGELOG
|
||||
|
||||
- R12. The root `CHANGELOG.md` shows distilled end-user notes per version. Per-package `CHANGELOG.md` files retain the structured changeset entries (summary + category + dev detail) as the developer-facing record.
|
||||
|
||||
---
|
||||
|
||||
## Key Technical Decisions
|
||||
|
||||
- **Keep the changeset versioning engine, impose structure on bodies.** The `@changesets/cli` correctly handles the fixed-group cross-package semver (`config.json` `fixed` array). Re-implementing that is real risk for zero versioning benefit. The plan imposes a structured *content* schema on changeset bodies and adds a linter; the versioning engine stays untouched.
|
||||
|
||||
- **Structured body format, not new frontmatter.** Changeset frontmatter (`---"@runfusion/fusion": minor---`) is consumed by the changesets tool and must stay machine-parseable. The structured content (`summary`, `category`, `dev`) lives in the body as labeled fields, parsed by a lightweight reader. This avoids fighting the changesets tool's frontmatter contract.
|
||||
|
||||
- **Distillation via `createFnAgent` with `tools: "readonly"`.** The PR-metadata generator (`packages/dashboard/src/pr-metadata-generator.ts`) already proves this exact pattern: single-shot model call, `onText` accumulation, settings-driven model resolution, graceful fallback on parse failure. The distillation module mirrors that shape. No new model infrastructure is needed.
|
||||
|
||||
- **Distillation runs in `release.mjs` (local) and as a post-version step in `version.yml` (CI).** Both paths share the same `scripts/lib/distill-release-notes.ts` module. In CI, the step runs after `changeset version` produces per-package CHANGELOGs but before the version PR commit, so curated notes ship with the version bump. Model credentials in CI come from a GitHub secret mapped to the existing settings model resolution.
|
||||
|
||||
- **Root CHANGELOG becomes the distilled view; per-package CHANGELOGs stay developer-facing.** The root `CHANGELOG.md` is the user-facing artifact (linked from the dashboard Update banner, GitHub Release). Per-package CHANGELOGs remain the developer/integrator record with structured changeset entries. `syncRootChangelog` is replaced by a distillation-aware sync that writes the distilled notes as the version's root section.
|
||||
|
||||
---
|
||||
|
||||
## High-Level Technical Design
|
||||
|
||||
### Changeset body schema
|
||||
|
||||
```
|
||||
---
|
||||
"@runfusion/fusion": minor
|
||||
---
|
||||
|
||||
summary: Add a Command Center productivity control for LOC backfills.
|
||||
category: feature
|
||||
dev: Uses the new `fn_backfill_loc` tool; settings key `commandCenter.locBackfill`.
|
||||
```
|
||||
|
||||
The body is parsed as labeled fields. `summary` and `category` are required; `dev` is optional. Any freeform text not matching a labeled field is treated as legacy content and triggers a linter warning.
|
||||
|
||||
### Release-time distillation flow
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
A[Pending changesets<br/>.changeset/*.md] --> B[changeset version<br/>bumps + per-pkg CHANGELOGs]
|
||||
B --> C[Parse structured summaries<br/>from versioned packages]
|
||||
C --> D{createFnAgent<br/>distill release notes}
|
||||
D -->|success| E[Distilled notes<br/>grouped by category]
|
||||
D -->|fail/timeout/parse| F[Fallback: bullet list<br/>from summary fields]
|
||||
E --> G[Write root CHANGELOG<br/>version section]
|
||||
F --> G
|
||||
G --> H[GitHub Release<br/>--notes-file]
|
||||
```
|
||||
|
||||
### Two release paths unified
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Local["Local release (release.mjs)"]
|
||||
L1[version] --> L2[distill] --> L3[root CHANGELOG] --> L4[gh release create]
|
||||
end
|
||||
subgraph CI["CI release (version.yml + release.yml)"]
|
||||
C1[changesets/action version] --> C2[distill step] --> C3[version PR with<br/>curated root CHANGELOG] --> C4[merge + tag] --> C5[release.yml uses<br/>CHANGELOG notes]
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Units
|
||||
|
||||
### U1. Structured changeset parser + schema
|
||||
|
||||
- **Goal:** Define the structured changeset body schema and ship a parser that extracts `summary`, `category`, and `dev` fields from a changeset markdown file, with legacy freeform fallback.
|
||||
- **Requirements:** R1, R2
|
||||
- **Dependencies:** none
|
||||
- **Files:**
|
||||
- `scripts/lib/changeset-schema.mjs` (new) — schema constants (categories, max summary length), parse and validate functions
|
||||
- `scripts/__tests__/changeset-schema.test.mjs` (new)
|
||||
- **Approach:** The parser reads a changeset `.md` file, splits frontmatter from body (reusing the `---` delimited convention), then extracts labeled fields (`summary:`, `category:`, `dev:`) from the body. Fields are parsed as `key: value` on the first line matching each label, with `dev` allowing multi-line content until the next labeled field or EOF. If no labeled fields are found, the entire body is treated as legacy `summary` (first line) with `category: internal` default and a `legacy: true` flag.
|
||||
- **Patterns to follow:** `readChangesetSummaries` in `scripts/release.mjs` (frontmatter parsing pattern); `parseAiResult` in `packages/dashboard/src/pr-metadata-generator.ts` (lenient parse with null fallback).
|
||||
- **Test scenarios:**
|
||||
- Happy path: parse a well-formed structured changeset with all three fields; assert each field is extracted correctly.
|
||||
- Multi-line `dev`: parse a changeset where `dev` spans multiple lines; assert full content captured.
|
||||
- Legacy freeform: parse an old-style changeset with a dense paragraph body and no labeled fields; assert `legacy: true`, `summary` is the first line, `category` defaults to `internal`.
|
||||
- Missing `category`: parse a changeset with `summary` but no `category`; assert validation flags it as missing.
|
||||
- Empty body: parse a changeset with frontmatter but empty body; assert graceful null return.
|
||||
- Summary over 120 chars: parse a changeset with an over-length `summary`; assert validation flags the violation.
|
||||
- **Verification:** Parser unit tests pass; parser correctly handles all three existing changeset shapes (structured, legacy paragraph, minimal one-liner) found in `.changeset/`.
|
||||
|
||||
### U2. Changeset linter
|
||||
|
||||
- **Goal:** Ship a linter script that validates every changeset in `.changeset/` against the structured schema, enforcing required fields, summary length, category enum, and frontmatter package scope. Wire it into the PR-check gate.
|
||||
- **Requirements:** R3, R4, R5
|
||||
- **Dependencies:** U1
|
||||
- **Files:**
|
||||
- `scripts/check-changeset-format.mjs` (new) — linter entrypoint
|
||||
- `scripts/__tests__/check-changeset-format.test.mjs` (new)
|
||||
- `.github/workflows/pr-checks.yml` (modify) — add changeset-format check step
|
||||
- `package.json` (modify) — add `check:changesets` script
|
||||
- `package.json` (modify) — add `check:changesets` to `test:gate` chain
|
||||
- **Approach:** The linter scans `.changeset/*.md` (excluding `README.md` and `config.json`), parses each via U1's parser, and validates: `summary` present and <= 120 chars, `category` is a valid enum value, frontmatter declares only `@runfusion/fusion`. Legacy changesets (no structured fields) produce a **warning** (exit 0) during the transition period; structurally invalid changesets (partial fields, bad category, over-length summary) produce **errors** (exit 1). The warning-vs-error threshold is a `--strict` flag so the transition can be tightened later.
|
||||
- **Patterns to follow:** `scripts/check-no-nohup.mjs`, `scripts/check-no-kill-4040.mjs` (existing lint-gate scripts that exit 1 on violation, integrated into `test:gate`).
|
||||
- **Test scenarios:**
|
||||
- Valid structured changeset passes with exit 0 and no warnings.
|
||||
- Legacy freeform changeset passes with exit 0 and a warning (transition mode).
|
||||
- Missing `category` on a structured changeset fails with exit 1 and names the file.
|
||||
- Over-length `summary` (121+ chars) fails with exit 1.
|
||||
- Invalid `category` value (e.g., `enhancement`) fails with exit 1 and lists valid values.
|
||||
- `--strict` flag causes legacy changesets to fail (exit 1) instead of warn.
|
||||
- Empty `.changeset/` directory (excluding config/README) passes with exit 0.
|
||||
- Frontmatter declaring a non-`@runfusion/fusion` package fails with exit 1.
|
||||
- **Verification:** `pnpm check:changesets` exits 0 against the current `.changeset/` directory (all existing entries are legacy and pass in transition mode). `pnpm test:gate` includes the check and passes.
|
||||
|
||||
### U3. AI distillation module
|
||||
|
||||
- **Goal:** Ship the distillation module that takes a version's parsed changesets and produces grouped, user-facing release notes via `createFnAgent`, with graceful fallback to a structured bullet list on any model failure.
|
||||
- **Requirements:** R6, R7, R8
|
||||
- **Dependencies:** U1
|
||||
- **Files:**
|
||||
- `scripts/lib/distill-release-notes.ts` (new) — distillation orchestrator
|
||||
- `scripts/__tests__/distill-release-notes.test.ts` (new)
|
||||
- **Approach:** The module accepts an array of parsed changeset entries (from U1), the target version, and an optional settings/model override. It builds a system prompt that instructs the model to produce grouped markdown release notes (sections: New, Fixed, Breaking, Performance, Security; omit empty sections), using only the `summary` fields as input and writing for a Fusion operator audience. It calls `createFnAgent` with `tools: "readonly"` and accumulates text via `onText`, mirroring `generatePrMetadata`. On success, the accumulated text is the release notes body. On any failure (model error, timeout, unparseable/empty output), the fallback builds a category-grouped bullet list directly from the structured `summary` fields — no model call. The module accepts an `AbortSignal` and timeout for release-script integration.
|
||||
- **Execution note:** Start with a failing test for the fallback path (no model available) to lock the graceful-degradation contract before implementing the model-call path.
|
||||
- **Patterns to follow:** `packages/dashboard/src/pr-metadata-generator.ts` (the canonical single-shot `createFnAgent` pattern: settings model resolution, `onText` accumulation, `AbortController` + timeout, try/finally `session.dispose()`, fallback on failure). `packages/engine/src/merger-ai.ts` (prompt-builder / verdict-parser separation for testability).
|
||||
- **Technical design (directional):**
|
||||
|
||||
```
|
||||
distillReleaseNotes({
|
||||
entries: ParsedChangeset[],
|
||||
version: string,
|
||||
settings?: Settings,
|
||||
signal?: AbortSignal,
|
||||
timeoutMs?: number,
|
||||
}): Promise<{ notes: string; source: "ai" | "fallback" }>
|
||||
```
|
||||
|
||||
The system prompt instructs: produce markdown grouped under `### New`, `### Fixed`, `### Breaking`, `### Performance`, `### Security`; use the `summary` text verbatim or lightly edited for grouping; omit empty sections; no internal class names or implementation detail; audience is a Fusion operator.
|
||||
|
||||
- **Test scenarios:**
|
||||
- Fallback path: call with `createFnAgent` stubbed to throw; assert returns bullet list grouped by category, `source: "fallback"`, exit 0.
|
||||
- Fallback path: call with model returning empty string; assert fallback bullet list.
|
||||
- Fallback path: call with model returning non-markdown garbage; assert fallback bullet list.
|
||||
- AI path: call with model stubbed to return grouped markdown; assert notes match, `source: "ai"`.
|
||||
- Timeout: call with `timeoutMs: 1` and a slow stub; assert fallback fires.
|
||||
- Abort: call with a pre-aborted signal; assert fallback fires immediately.
|
||||
- Category grouping in fallback: pass entries with categories `feature`, `fix`, `breaking`; assert fallback groups them under correct headings and omits empty sections.
|
||||
- Legacy entries: pass entries with `category: internal` (legacy default); assert they appear in an "Internal" section or are omitted per audience rule.
|
||||
- Single entry: pass one entry; assert notes are well-formed with one bullet.
|
||||
- **Verification:** Module unit tests pass with both stubbed model (AI path) and forced-failure (fallback path). Module produces valid markdown for the current pending changesets when run manually with a real model.
|
||||
|
||||
### U4. Release script integration (local path)
|
||||
|
||||
- **Goal:** Wire the distillation module into `release.mjs` so the local release produces distilled notes for both the root `CHANGELOG.md` and the GitHub Release.
|
||||
- **Requirements:** R9, R12
|
||||
- **Dependencies:** U1, U3
|
||||
- **Files:**
|
||||
- `scripts/release.mjs` (modify) — replace `syncRootChangelog` + `extractVersionNotes` with distillation-aware versions
|
||||
- `scripts/lib/sync-root-changelog.mjs` (new, extracted from `release.mjs`) — refactored root CHANGELOG sync that accepts a distilled-notes override for the version section
|
||||
- `scripts/__tests__/sync-root-changelog.test.mjs` (new)
|
||||
- **Approach:** After `pnpm release:version` runs (which bumps versions and writes per-package CHANGELOGs), the script reads the versioned changesets (now deleted from `.changeset/` by `changeset version`), so the changeset content must be captured **before** `changeset version` runs. Add a pre-version capture step that reads and parses all pending changesets via U1, then after versioning, passes the parsed entries to U3's `distillReleaseNotes`. The distilled notes replace the version's section in the root `CHANGELOG.md` (per-package CHANGELOGs are untouched — they carry the structured entries). The GitHub Release uses the distilled notes directly via `--notes-file`. Extract `syncRootChangelog` into its own module so it can accept the distilled-notes override.
|
||||
- **Patterns to follow:** The existing `release.mjs` flow ordering (version → sync → build → commit → publish → tag → release). The `readChangesetSummaries` function already captures pre-version changeset content; extend it to use U1's parser.
|
||||
- **Test scenarios:**
|
||||
- `syncRootChangelog` with distilled override: pass a version, existing root CHANGELOG content, and distilled notes; assert the version section is replaced with the distilled notes while other versions are preserved.
|
||||
- `syncRootChangelog` without override (fallback): pass no distilled notes; assert behavior matches current aggregation (backward compat for dry-runs without model).
|
||||
- Pre-version capture: mock `.changeset/` with structured entries; assert entries are captured before `changeset version` deletes them.
|
||||
- Pre-version capture with legacy entries: assert legacy entries are captured with `legacy: true` and still feed distillation.
|
||||
- **Verification:** `pnpm release --dry-run` shows the captured changesets and proposed distilled notes without making changes. A real release produces a root `CHANGELOG.md` with the distilled version section and a GitHub Release with matching notes.
|
||||
|
||||
### U5. CI workflow integration
|
||||
|
||||
- **Goal:** Wire distillation into the CI version workflow (`version.yml`) and update the binary release workflow (`release.yml`) to use the curated notes.
|
||||
- **Requirements:** R10, R11
|
||||
- **Dependencies:** U3, U4
|
||||
- **Files:**
|
||||
- `.github/workflows/version.yml` (modify) — add a post-version distillation step between `changeset version` and the version PR commit
|
||||
- `.github/workflows/release.yml` (modify) — replace `generate_release_notes: true` with `--notes-file` reading the root CHANGELOG version section
|
||||
- `scripts/ci-distill-release-notes.mjs` (new) — CI entrypoint that resolves model credentials from GitHub secrets, runs distillation, and writes the root CHANGELOG
|
||||
- **Approach:** In `version.yml`, after `changesets/action` runs the version step (which calls `pnpm release:version`), add a new step that runs `node scripts/ci-distill-release-notes.mjs --version <version>`. This script reads the just-versioned per-package CHANGELOGs, extracts the structured entries for the new version, calls U3's distillation module with model settings resolved from a GitHub secret (`FUSION_RELEASE_MODEL_API_KEY` or similar, mapped through the existing settings model resolution), and writes the distilled notes into the root `CHANGELOG.md`. The version PR then carries curated notes. In `release.yml`, replace `generate_release_notes: true` with a step that extracts the version section from the root `CHANGELOG.md` (via `extractVersionNotes`) and passes it as `--notes-file`, so the GitHub Release matches the curated notes.
|
||||
- **Patterns to follow:** The existing `version.yml` `changesets/action` integration; the `release.yml` `softprops/action-gh-release` usage.
|
||||
- **Test scenarios:**
|
||||
- CI distillation entrypoint: mock per-package CHANGELOGs with structured entries; assert the script produces distilled root CHANGELOG section.
|
||||
- CI distillation with no model secret: assert the script falls back gracefully (bullet list) and does not fail the workflow.
|
||||
- `release.yml` notes extraction: given a root CHANGELOG with a distilled version section, assert `extractVersionNotes` returns the correct content for `--notes-file`.
|
||||
- `release.yml` notes extraction: version not found in CHANGELOG; assert fallback string is used.
|
||||
- **Verification:** `version.yml` workflow run (manual dispatch) produces a version PR with a distilled root CHANGELOG section. `release.yml` GitHub Release body matches the root CHANGELOG version section.
|
||||
|
||||
### U6. Agent guidance + documentation update
|
||||
|
||||
- **Goal:** Update all documentation and agent-facing guidance so the agent fleet and human contributors author changesets in the new structured format.
|
||||
- **Requirements:** R1, R3
|
||||
- **Dependencies:** U1, U2
|
||||
- **Files:**
|
||||
- `AGENTS.md` (modify) — update the "Finalizing Changesets" section with the structured format, field definitions, category enum, and examples
|
||||
- `RELEASING.md` (modify) — update the changeset authoring section with the new format
|
||||
- `docs/contributing.md` (modify) — update the changeset reference
|
||||
- `.changeset/README.md` (new) — template + format reference that `pnpm changeset` consumers see
|
||||
- **Approach:** The AGENTS.md "Finalizing Changesets" section currently says "add a changeset" with a bump-type table. Expand it with the structured body schema: required `summary` (one line, user-facing, max 120 chars), required `category` (enum), optional `dev` (developer detail). Provide before/after examples showing the transformation from dense paragraph to structured fields. Note the linter enforcement and the transition period (legacy changesets warn, don't fail). Add a `.changeset/README.md` that `pnpm changeset` surfaces as the format guide.
|
||||
- **Patterns to follow:** The existing AGENTS.md "Finalizing Changesets" section structure (bump types, rules table).
|
||||
- **Test expectation:** none — documentation-only unit.
|
||||
- **Verification:** AGENTS.md guidance matches the U1 schema and U2 linter rules exactly. A new contributor reading the docs can author a valid structured changeset without further guidance.
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
### Deferred for later
|
||||
|
||||
- Backfilling the existing 10,000-line root `CHANGELOG.md` history into the distilled format. Only future releases get the new treatment; historical versions retain their current content.
|
||||
- A dashboard surface for browsing release notes in-app (beyond the existing Update banner link to GitHub). The plan covers the notes pipeline, not a new UI surface.
|
||||
- Per-package CHANGELOG distillation (the per-package files keep structured entries; only the root CHANGELOG is distilled).
|
||||
- Migration of the `@changesets/cli` tool itself to a custom versioning system.
|
||||
|
||||
### Outside this product's identity
|
||||
|
||||
- Replacing the changeset versioning engine with a commit-conventional or PR-derived changelog generator. The confirmed direction is structured changesets + distillation, not deriving from PRs.
|
||||
|
||||
---
|
||||
|
||||
## Risks & Dependencies
|
||||
|
||||
- **CI model credentials.** Distillation in `version.yml` requires a model API key available as a GitHub secret. If this is not configured, CI distillation must fall back gracefully (bullet list from summaries). The local release path is unaffected (uses the operator's local settings). This is the main operational dependency.
|
||||
- **`createFnAgent` from a root script.** No existing root-level script imports `@fusion/*` packages (they are TS source resolved via workspace aliases). The distillation module is written in TypeScript and run via `tsx` (already a devDependency). The CI entrypoint is `.mjs` and shells to the TS module via `tsx`. If workspace resolution is problematic in the release context, the fallback is to shell out to a Fusion CLI one-shot (if one exists) or inline the model call. This is an execution-time unknown to resolve during U3.
|
||||
- **Changeset capture timing.** `changeset version` deletes changeset files after consuming them. The pre-version capture step (U4) must read and parse changesets before `changeset version` runs. If the capture step is missed, the per-package CHANGELOGs (which contain the aggregated entries) can serve as a secondary source. The implementation should handle both paths.
|
||||
- **Transition period ambiguity.** During the transition, a release may contain a mix of structured and legacy changesets. The distillation module and fallback must handle mixed input gracefully (legacy entries use the first-line-as-summary default).
|
||||
|
||||
---
|
||||
|
||||
## System-Wide Impact
|
||||
|
||||
- **Agent fleet behavior.** Every AI agent that completes a task with a changeset must author the new structured format. This is a behavioral change enforced by the linter and documented in AGENTS.md. The transition period prevents immediate breakage.
|
||||
- **Release pipeline.** Both release paths (local and CI) gain a model-call step. The local path adds ~10-30 seconds for distillation. The CI path adds a step to the version workflow. Both degrade gracefully on model failure.
|
||||
- **External consumers.** The root `CHANGELOG.md` and GitHub Release notes change shape (from raw changeset aggregation to curated, grouped notes). Users who parse the CHANGELOG programmatically may need to adjust. The per-package CHANGELOGs retain the changesets structure for npm consumers.
|
||||
|
||||
---
|
||||
|
||||
## Documentation / Operational Notes
|
||||
|
||||
- Operators running local releases (`pnpm release`) get distillation automatically using their configured model settings — no additional setup.
|
||||
- CI distillation requires a GitHub secret for the model API key. Document the required secret name and settings shape in `RELEASING.md`.
|
||||
- The `--strict` flag on the changeset linter allows tightening the transition: once all agents produce structured changesets, flip the gate from warning to error on legacy format.
|
||||
- The distillation system prompt and category mappings are defined in `scripts/lib/distill-release-notes.ts` and can be tuned without code changes beyond the prompt string.
|
||||
@@ -1,45 +1,108 @@
|
||||
/*
|
||||
* FNXC:Changelog 2026-06-24-15:30:
|
||||
* Release-notes distiller. Takes parsed changeset entries and produces
|
||||
* grouped, end-user-facing release notes organized by category. The
|
||||
* deterministic mode renders a clean category-grouped bullet list from
|
||||
* structured summaries. The AI mode (optional, added in U4) calls
|
||||
* createFnAgent for polished prose. When the AI call fails or is
|
||||
* unavailable, the deterministic output is the fallback so a model
|
||||
* outage never blocks a release.
|
||||
* Release-notes distillation module. Transforms parsed changeset entries
|
||||
* into grouped, end-user-facing release notes. The deterministic fallback
|
||||
* builds a category-grouped bullet list directly from the structured
|
||||
* `summary` fields — no model call. When a model is available, the prompt
|
||||
* and system prompt defined here can be used to produce curated, polished
|
||||
* notes via `createFnAgent`.
|
||||
*
|
||||
* Audience is Fusion operators: behavior, fixes, what changed — minimal
|
||||
* internals. The `dev` field is preserved in per-package CHANGELOGs but
|
||||
* excluded from distilled release notes by default.
|
||||
*/
|
||||
|
||||
import { CATEGORIES, CATEGORY_HEADINGS } from "./changeset-schema.mjs";
|
||||
|
||||
/**
|
||||
* Deterministic distillation: groups parsed changeset entries by category
|
||||
* and renders clean markdown release notes.
|
||||
* System prompt for AI distillation via `createFnAgent`.
|
||||
* Instructs the model to produce grouped markdown release notes for a
|
||||
* Fusion operator audience, using only the `summary` fields as input.
|
||||
*/
|
||||
export const DISTILLATION_SYSTEM_PROMPT = [
|
||||
"You are a release-notes writer for Fusion, a model-agnostic AI agent orchestration product.",
|
||||
"Your audience is Fusion operators — developers using the product, not its internals.",
|
||||
"Produce clean, grouped markdown release notes from the provided changeset entries.",
|
||||
"Group under these headings (omit empty sections):",
|
||||
" ### New (features)",
|
||||
" ### Fixed (bug fixes)",
|
||||
" ### Breaking (breaking changes)",
|
||||
" ### Security (security fixes)",
|
||||
" ### Performance (performance improvements)",
|
||||
" ### Internal (internal-only changes)",
|
||||
"Rules:",
|
||||
"- Use the `summary` text verbatim or lightly edited for clarity and grouping.",
|
||||
"- Do NOT include internal class names, file paths, or implementation detail.",
|
||||
"- Do NOT include the `dev` field content unless it is user-relevant migration guidance.",
|
||||
"- Write one bullet per entry, prefixed with `- `.",
|
||||
"- Omit empty sections entirely.",
|
||||
"- Do NOT add a title or version heading — only the grouped sections.",
|
||||
"- Respond with markdown only, no preamble or explanation.",
|
||||
].join("\n");
|
||||
|
||||
/**
|
||||
* Build the user-facing prompt for AI distillation.
|
||||
* Lists each entry as `[N] category: X / summary: Y / dev: Z`.
|
||||
*
|
||||
* Categories are rendered in display order (feature → fix → breaking →
|
||||
* security → performance → internal). Empty categories are omitted.
|
||||
* The `internal` category is included only when it carries entries, since
|
||||
* operators generally don't need internal-only changes surfaced — but it
|
||||
* is not suppressed entirely because some releases are internal-only.
|
||||
* @param {Array<{summary: string, category: string, dev?: string, legacy?: boolean}>} entries
|
||||
* @returns {string}
|
||||
*/
|
||||
export function buildDistillationPrompt(entries) {
|
||||
const lines = ["Produce release notes from these changeset entries:\n"];
|
||||
entries.forEach((entry, i) => {
|
||||
const num = i + 1;
|
||||
lines.push(`[${num}]`);
|
||||
lines.push(` category: ${entry.category}`);
|
||||
lines.push(` summary: ${entry.summary}`);
|
||||
if (entry.dev) {
|
||||
lines.push(` dev: ${entry.dev}`);
|
||||
}
|
||||
lines.push("");
|
||||
});
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Deterministic fallback: build category-grouped release notes directly
|
||||
* from the structured `summary` fields — no model call.
|
||||
*
|
||||
* @param {Array<{summary: string, category: string, dev?: string, legacy: boolean}>} entries
|
||||
* @param {string} version
|
||||
* Used when:
|
||||
* - The model call fails, times out, or returns unparseable output
|
||||
* - No model is configured (CI without model secret)
|
||||
* - As a pre-model preview in dry-runs
|
||||
*
|
||||
* @param {Array<{summary: string, category: string, legacy?: boolean}>} entries
|
||||
* @param {string} version - Target version string (e.g. "0.47.0")
|
||||
* @returns {{notes: string, source: "deterministic"}}
|
||||
*/
|
||||
export function distillDeterministic(entries, version) {
|
||||
if (!entries || entries.length === 0) {
|
||||
return { notes: `No changes in v${version}.`, source: "deterministic" };
|
||||
return {
|
||||
notes: `No changes in v${version}.`,
|
||||
source: "deterministic",
|
||||
};
|
||||
}
|
||||
|
||||
const grouped = groupByCategory(entries);
|
||||
// Group entries by category, preserving entry order within each group.
|
||||
const groups = new Map();
|
||||
for (const cat of CATEGORIES) {
|
||||
groups.set(cat, []);
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
const cat = groups.has(entry.category) ? entry.category : "internal";
|
||||
groups.get(cat).push(entry.summary);
|
||||
}
|
||||
|
||||
// Build sections in display order, omitting empty categories.
|
||||
const sections = [];
|
||||
for (const cat of CATEGORIES) {
|
||||
const summaries = groups.get(cat);
|
||||
if (summaries.length === 0) continue;
|
||||
|
||||
for (const category of CATEGORIES) {
|
||||
const items = grouped.get(category);
|
||||
if (!items || items.length === 0) continue;
|
||||
|
||||
const heading = CATEGORY_HEADINGS[category];
|
||||
const bullets = items.map((entry) => `- ${entry.summary}`);
|
||||
sections.push(`### ${heading}\n\n${bullets.join("\n")}`);
|
||||
const heading = CATEGORY_HEADINGS[cat];
|
||||
const bullets = summaries.map((s) => `- ${s}`).join("\n");
|
||||
sections.push(`### ${heading}\n\n${bullets}`);
|
||||
}
|
||||
|
||||
return {
|
||||
@@ -47,50 +110,3 @@ export function distillDeterministic(entries, version) {
|
||||
source: "deterministic",
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Group entries by category, preserving input order within each group.
|
||||
*/
|
||||
function groupByCategory(entries) {
|
||||
const grouped = new Map();
|
||||
for (const category of CATEGORIES) {
|
||||
grouped.set(category, []);
|
||||
}
|
||||
for (const entry of entries) {
|
||||
const cat = CATEGORIES.includes(entry.category) ? entry.category : "internal";
|
||||
grouped.get(cat)?.push(entry);
|
||||
}
|
||||
return grouped;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the context prompt for AI distillation. This is used by the AI
|
||||
* mode (U4) when calling createFnAgent. Exported separately so the prompt
|
||||
* can be tested without a model call.
|
||||
*
|
||||
* @param {Array<{summary: string, category: string, dev?: string, legacy: boolean}>} entries
|
||||
* @returns {string}
|
||||
*/
|
||||
export function buildDistillationPrompt(entries) {
|
||||
const lines = entries.map((entry, i) => {
|
||||
const parts = [`[${i + 1}] category: ${entry.category}`, `summary: ${entry.summary}`];
|
||||
if (entry.dev) {
|
||||
parts.push(`dev: ${entry.dev}`);
|
||||
}
|
||||
return parts.join("\n");
|
||||
});
|
||||
return lines.join("\n\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* System prompt for AI distillation. Exported for testing and tuning.
|
||||
*/
|
||||
export const DISTILLATION_SYSTEM_PROMPT = [
|
||||
"You are a release notes writer for Fusion, an AI agent orchestration tool.",
|
||||
"Your audience is Fusion operators (developers using the tool), not internal engineers.",
|
||||
"Given structured changeset entries, produce grouped markdown release notes.",
|
||||
"Group under these headings: ### New, ### Fixed, ### Breaking, ### Performance, ### Security.",
|
||||
"Omit empty sections. Use bullet points.",
|
||||
"Write clear, concise, user-facing summaries. No internal class names or implementation detail.",
|
||||
"Output only the markdown release notes, no preamble or explanation.",
|
||||
].join("\n");
|
||||
|
||||
Reference in New Issue
Block a user