FN-6694: add workflow reliability release-check harness

Add an on-demand executable release-check lane for custom workflow reliability signoff. - Add a manifest-backed release-check runner with dry-run, JSON output, validation, and targeted Vitest command planning. - Define the custom workflow reliability checklist seams and cover runner validation, planning, reporting, and failure behavior with node tests. - Document the QA signoff command and manifest mapping while keeping the lane out of the merge gate. Files changed: docs/custom-workflow-reliability-acceptance-map.md | 29 +- docs/testing.md | 3 + package.json | 1 + .../workflow-reliability-release-check.test.mjs | 138 +++++++++ .../lib/workflow-reliability-release-check.json | 102 +++++++ scripts/workflow-reliability-release-check.mjs | 312 +++++++++++++++++++++ 6 files changed, 579 insertions(+), 6 deletions(-) Fusion-Task-Id: FN-6694 Fusion-Task-Lineage: 21d45bb5-6c45-4cb7-8b29-9eaea8b29ad4
2026-06-19 05:13:39 -07:00
parent 3566cf8a1d
commit 3383a095b7
6 changed files with 579 additions and 6 deletions
--- a/docs/custom-workflow-reliability-acceptance-map.md
+++ b/docs/custom-workflow-reliability-acceptance-map.md
@@ -116,10 +116,27 @@ The following journeys are intentionally out of scope for the MVP reliability ba

 ## Release-check checklist

-Before claiming the custom workflow system is reliable for goal **G-MPW67VQR-0001-97S3**, QA or engineering should be able to demonstrate:
+<!--
+FNXC:CustomWorkflowReliability 2026-06-19-00:00:
+FN-6694 made this release checklist executable. QA/release signoff should now cite the harness output and manifest-backed seam mapping rather than relying on prose-only spot checks.
+-->

- A custom workflow can be authored/imported, rejected on invalid IR, saved, discovered, selected, and reloaded.
- A task can execute the selected workflow through runtime primitives, and explicit missing custom workflow IDs fail closed.
- Gate/advisory/readonly/`REVISE`/required-artifact behavior is observable in task state, workflow results, task documents, and logs.
- `autoMerge:false`, hard-cancel, file-scope, and recovery invariants are preserved under custom workflow execution.
- Engine/scheduler restart preserves workflow selection and progress, and any recovery emits typed run-audit evidence instead of silent lifecycle mutation.
+Before claiming the custom workflow system is reliable for goal **G-MPW67VQR-0001-97S3**, QA or engineering should run the executable release-check harness:
+
+```bash
+pnpm test:workflow-release-check          # run the targeted manifest-listed seams and emit text PASS/FAIL evidence
+pnpm test:workflow-release-check --json   # emit the same item/seam evidence as machine-readable JSON
+pnpm test:workflow-release-check --dry-run # validate the manifest and print planned commands without running Vitest
+```
+
+The source of truth for the checklist-to-seam mapping is [`scripts/lib/workflow-reliability-release-check.json`](../scripts/lib/workflow-reliability-release-check.json). The runner validates that every referenced file exists, groups the seams into targeted package-scoped Vitest commands, and exits non-zero if the manifest is invalid or any required item fails. It is intentionally an on-demand QA/release lane, not a merge-gate expansion.
+
+| Release-check item | Manifest ID | Automated evidence seams |
+|---|---|---|
+| A custom workflow can be authored/imported, rejected on invalid IR, saved, discovered, selected, and reloaded. | `author-import-save-discover-reload` | `packages/core/src/__tests__/workflow-definition-store.test.ts`; `packages/dashboard/src/routes/__tests__/workflow-import-export.test.ts`; `packages/dashboard/src/routes/__tests__/workflow-design-route.test.ts`; `packages/core/src/__tests__/workflow-selection-store.test.ts` |
+| A task can execute the selected workflow through runtime primitives, and explicit missing custom workflow IDs fail closed. | `selected-workflow-execution-fail-closed` | `packages/core/src/__tests__/workflow-selection-store.test.ts`; `packages/engine/src/__tests__/workflow-task-runtime.test.ts` |
+| Gate/advisory/readonly/`REVISE`/required-artifact behavior is observable in task state, workflow results, task documents, and logs. | `gate-advisory-readonly-revise-required-artifact` | `packages/engine/src/__tests__/workflow-malformed-verdict-gate.test.ts`; `packages/engine/src/__tests__/workflow-required-artifact-gate.test.ts`; `packages/engine/src/__tests__/workflow-step-readonly-allowlist.test.ts`; `packages/engine/src/__tests__/executor-workflow-revision-scope.test.ts` |
+| `autoMerge:false`, hard-cancel, file-scope, and recovery invariants are preserved under custom workflow execution. | `automerge-hard-cancel-file-scope-recovery` | `packages/engine/src/__tests__/reliability-interactions/workflow-and-file-scope.test.ts`; `packages/engine/src/__tests__/reliability-interactions/workflow-interpreter-cutover.test.ts`; `packages/engine/src/__tests__/self-healing-custom-workflow-recovery.test.ts` |
+| Engine/scheduler restart preserves workflow selection and progress, and any recovery emits typed run-audit evidence instead of silent lifecycle mutation. | `restart-selection-progress-run-audit` | `packages/core/src/__tests__/workflow-restart-durability.test.ts`; `packages/engine/src/__tests__/self-healing-custom-workflow-recovery.test.ts` |
+
+Manual-only checks: **none currently deferred**. If a future release-check item cannot be automated, add it to the manifest's `manual` array with a non-empty `automationDeferredReason`, label it here, and file/link a focused follow-up after confirming there is no duplicate task.
--- a/docs/testing.md
+++ b/docs/testing.md
@@ -40,6 +40,9 @@ pnpm verify:workspace  # deep opt-in verification: lint -> test:full -> build (N

 `pnpm test:full` runs each package's default test script with capped worker fanout (`FUSION_TEST_TOTAL_WORKERS=4 FUSION_TEST_CONCURRENCY=2 pnpm -r --workspace-concurrency=2 test`). Do not casually raise worker counts; dashboard/jsdom and integration-heavy packages destabilize when oversubscribed. Use `VITEST_MAX_WORKERS=<n>` only for targeted package-level investigation.

+<!-- FNXC:CustomWorkflowReliability 2026-06-19-00:00: FN-6694 adds an executable custom-workflow reliability release-check lane for QA signoff, but it must stay out of the merge gate so reliability evidence does not inflate every PR's wall-time. -->
+Custom workflow reliability release signoff has a dedicated on-demand lane: `pnpm test:workflow-release-check` runs the manifest-listed targeted seams from `scripts/lib/workflow-reliability-release-check.json`, while `--dry-run` validates the manifest and prints planned commands and `--json` emits machine-readable item/seam evidence. This lane is **not** part of the merge gate and should not be added to `test:gate` or the `engine-core` allow-list.
+
 <!-- FNXC:iOSAcceptance 2026-06-18-17:25: Terminal acceptance gates that depend on real mobile Safari must use the credential-driven real-iOS surface runbook instead of treating desktop WebKit or jsdom as evidence. -->
 Terminal acceptance tasks that require real mobile Safari should use [`docs/ios-acceptance.md`](./ios-acceptance.md) for the `--check` run-vs-NO-OP probe, credential wiring, and physical/cloud real-iOS evidence workflow.

--- a/package.json
+++ b/package.json
@@ -32,6 +32,7 @@
    "build:exe:all": "pnpm build && pnpm --filter @runfusion/fusion build:exe:all",
    "test": "node scripts/test-changed.mjs",
    "test:scripts": "node --test scripts/__tests__/*.test.mjs",
+    "test:workflow-release-check": "node scripts/workflow-reliability-release-check.mjs",
    "fn:cache-stats": "node scripts/cache-stats.mjs",
    "test:full": "node scripts/test-changed.mjs --full --no-cache && pnpm --filter @fusion/engine test:slow",
    "test:velocity": "node scripts/test-velocity-baseline.mjs",
--- a/scripts/tests/workflow-reliability-release-check.test.mjs
+++ b/scripts/tests/workflow-reliability-release-check.test.mjs
@@ -0,0 +1,138 @@
+import test from "node:test";
+import assert from "node:assert/strict";
+import path from "node:path";
+import { existsSync } from "node:fs";
+import { fileURLToPath } from "node:url";
+import {
+  DEFAULT_MANIFEST_PATH,
+  loadManifest,
+  planCommands,
+  renderReport,
+  summarize,
+  validateManifest,
+} from "../workflow-reliability-release-check.mjs";
+
+const currentFilePath = fileURLToPath(import.meta.url);
+const repoRoot = path.resolve(path.dirname(currentFilePath), "../..");
+
+const expectedChecklistIds = [
+  "author-import-save-discover-reload",
+  "selected-workflow-execution-fail-closed",
+  "gate-advisory-readonly-revise-required-artifact",
+  "automerge-hard-cancel-file-scope-recovery",
+  "restart-selection-progress-run-audit",
+];
+
+function clone(value) {
+  return JSON.parse(JSON.stringify(value));
+}
+
+test("manifest parses, covers all five release-check items, and references existing seam files", () => {
+  const manifest = loadManifest(DEFAULT_MANIFEST_PATH, { rootDir: repoRoot });
+
+  assert.equal(manifest.version, 1);
+  assert.equal(typeof manifest._comment, "string");
+  assert.deepEqual(manifest.checklist.map((item) => item.id), expectedChecklistIds);
+  assert.equal(manifest.manual.length, 0);
+
+  for (const item of manifest.checklist) {
+    assert.equal(typeof item.title, "string");
+    assert.equal(typeof item.journey, "string");
+    assert.ok(item.seams.length >= 1, `${item.id} should map to at least one automated seam`);
+    for (const seam of item.seams) {
+      assert.match(seam.package, /^@fusion\/(core|dashboard|engine)$/);
+      assert.ok(existsSync(path.join(repoRoot, seam.file)), `${seam.file} should exist`);
+    }
+  }
+
+  assert.deepEqual(validateManifest(manifest, { repoRoot, existsSync }), { ok: true, errors: [] });
+});
+
+test("planned commands are targeted package-scoped vitest invocations", () => {
+  const manifest = loadManifest(DEFAULT_MANIFEST_PATH, { rootDir: repoRoot });
+  const commands = planCommands(manifest);
+
+  assert.deepEqual(commands.map((command) => command.package), ["@fusion/core", "@fusion/dashboard", "@fusion/engine"]);
+  for (const command of commands) {
+    assert.equal(command.command, "pnpm");
+    assert.deepEqual(command.args.slice(0, 5), ["--filter", command.package, "exec", "vitest", "run"]);
+    assert.ok(command.args.includes("--silent=passed-only"));
+    assert.ok(command.args.includes("--reporter=dot"));
+    assert.ok(command.files.length >= 1);
+    for (const file of command.files) assert.ok(file.startsWith("packages/"));
+  }
+});
+
+test("validateManifest fails closed on dangling seam files and uncovered checklist items", () => {
+  const manifest = loadManifest(DEFAULT_MANIFEST_PATH, { rootDir: repoRoot });
+  const dangling = clone(manifest);
+  dangling.checklist[0].seams[0].file = "packages/core/src/__tests__/missing-workflow-release-check.test.ts";
+
+  const danglingResult = validateManifest(dangling, { repoRoot, existsSync: (candidate) => !String(candidate).includes("missing-workflow-release-check") });
+  assert.equal(danglingResult.ok, false);
+  assert.match(danglingResult.errors.join("\n"), /file does not exist/);
+
+  const uncovered = clone(manifest);
+  uncovered.checklist[1].seams = [];
+  const uncoveredResult = validateManifest(uncovered, { repoRoot, existsSync: () => true });
+  assert.equal(uncoveredResult.ok, false);
+  assert.match(uncoveredResult.errors.join("\n"), /must have at least one seam or a manual automationDeferredReason/);
+});
+
+test("validateManifest rejects unknown packages and accepts explicit manual deferrals", () => {
+  const manifest = loadManifest(DEFAULT_MANIFEST_PATH, { rootDir: repoRoot });
+  const unknownPackage = clone(manifest);
+  unknownPackage.checklist[0].seams[0].package = "@fusion/unknown";
+
+  const unknownPackageResult = validateManifest(unknownPackage, { repoRoot, existsSync: () => true });
+  assert.equal(unknownPackageResult.ok, false);
+  assert.match(unknownPackageResult.errors.join("\n"), /unknown package/);
+
+  const manual = clone(manifest);
+  manual.checklist[0].seams = [];
+  manual.manual = [{
+    id: manual.checklist[0].id,
+    title: manual.checklist[0].title,
+    automationDeferredReason: "Requires a human-only external signoff artifact.",
+  }];
+  assert.deepEqual(validateManifest(manual, { repoRoot, existsSync: () => true }), { ok: true, errors: [] });
+});
+
+test("summarize and renderReport roll up pass, fail, manual, text, and stable JSON", () => {
+  const summary = summarize([
+    {
+      id: "passed-item",
+      title: "Passed item",
+      journey: "A passing synthetic item",
+      seams: [{ package: "@fusion/core", file: "one.test.ts", status: "PASS" }],
+      manual: null,
+    },
+    {
+      id: "failed-item",
+      title: "Failed item",
+      journey: "A failing synthetic item",
+      seams: [{ package: "@fusion/engine", file: "two.test.ts", status: "FAIL", exitCode: 1 }],
+      manual: null,
+    },
+    {
+      id: "manual-item",
+      title: "Manual item",
+      journey: "A manual synthetic item",
+      seams: [],
+      manual: { id: "manual-item", title: "Manual item", automationDeferredReason: "Human inspection only." },
+    },
+  ]);
+
+  assert.equal(summary.ok, false);
+  assert.deepEqual(summary.counts, { pass: 1, fail: 1, manual: 1, total: 3 });
+  assert.deepEqual(summary.items.map((item) => item.status), ["PASS", "FAIL", "MANUAL"]);
+
+  const text = renderReport(summary);
+  assert.match(text, /Overall: FAIL \(1 passed, 1 failed, 1 manual, 3 total\)/);
+  assert.match(text, /FAIL: failed-item/);
+  assert.match(text, /MANUAL: Human inspection only\./);
+
+  const json = renderReport(summary, { json: true });
+  assert.deepEqual(JSON.parse(json), summary);
+  assert.ok(json.startsWith('{\n  "ok": false,'));
+});
--- a/scripts/lib/workflow-reliability-release-check.json
+++ b/scripts/lib/workflow-reliability-release-check.json
@@ -0,0 +1,102 @@
+{
+  "version": 1,
+  "_comment": "FNXC:CustomWorkflowReliability 2026-06-19-00:00: FN-6694 requires this manifest to encode goal G-MPW67VQR-0001-97S3's release-check checklist as objective, runnable seam evidence instead of prose-only QA signoff.",
+  "checklist": [
+    {
+      "id": "author-import-save-discover-reload",
+      "title": "Custom workflow authoring, import, invalid rejection, save, discovery, selection, and reload",
+      "journey": "A custom workflow can be authored/imported, rejected on invalid IR, saved, discovered, selected, and reloaded.",
+      "seams": [
+        {
+          "package": "@fusion/core",
+          "file": "packages/core/src/__tests__/workflow-definition-store.test.ts"
+        },
+        {
+          "package": "@fusion/dashboard",
+          "file": "packages/dashboard/src/routes/__tests__/workflow-import-export.test.ts"
+        },
+        {
+          "package": "@fusion/dashboard",
+          "file": "packages/dashboard/src/routes/__tests__/workflow-design-route.test.ts"
+        },
+        {
+          "package": "@fusion/core",
+          "file": "packages/core/src/__tests__/workflow-selection-store.test.ts"
+        }
+      ]
+    },
+    {
+      "id": "selected-workflow-execution-fail-closed",
+      "title": "Selected workflow execution and fail-closed missing custom workflow IDs",
+      "journey": "A task can execute the selected workflow through runtime primitives, and explicit missing custom workflow IDs fail closed.",
+      "seams": [
+        {
+          "package": "@fusion/core",
+          "file": "packages/core/src/__tests__/workflow-selection-store.test.ts"
+        },
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/workflow-task-runtime.test.ts"
+        }
+      ]
+    },
+    {
+      "id": "gate-advisory-readonly-revise-required-artifact",
+      "title": "Gate, advisory, readonly, REVISE, and required-artifact observability",
+      "journey": "Gate/advisory/readonly/REVISE/required-artifact behavior is observable in task state, workflow results, task documents, and logs.",
+      "seams": [
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/workflow-malformed-verdict-gate.test.ts"
+        },
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/workflow-required-artifact-gate.test.ts"
+        },
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/workflow-step-readonly-allowlist.test.ts"
+        },
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/executor-workflow-revision-scope.test.ts"
+        }
+      ]
+    },
+    {
+      "id": "automerge-hard-cancel-file-scope-recovery",
+      "title": "autoMerge:false, hard-cancel, file-scope, and recovery invariants",
+      "journey": "autoMerge:false, hard-cancel, file-scope, and recovery invariants are preserved under custom workflow execution.",
+      "seams": [
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/reliability-interactions/workflow-and-file-scope.test.ts"
+        },
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/reliability-interactions/workflow-interpreter-cutover.test.ts"
+        },
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/self-healing-custom-workflow-recovery.test.ts"
+        }
+      ]
+    },
+    {
+      "id": "restart-selection-progress-run-audit",
+      "title": "Restart durability for workflow selection, progress, and recovery evidence",
+      "journey": "Engine/scheduler restart preserves workflow selection and progress, and any recovery emits typed run-audit evidence instead of silent lifecycle mutation.",
+      "seams": [
+        {
+          "package": "@fusion/core",
+          "file": "packages/core/src/__tests__/workflow-restart-durability.test.ts"
+        },
+        {
+          "package": "@fusion/engine",
+          "file": "packages/engine/src/__tests__/self-healing-custom-workflow-recovery.test.ts"
+        }
+      ]
+    }
+  ],
+  "manual": []
+}
--- a/scripts/workflow-reliability-release-check.mjs
+++ b/scripts/workflow-reliability-release-check.mjs
@@ -0,0 +1,312 @@
+#!/usr/bin/env node
+
+import { existsSync, readFileSync } from "node:fs";
+import path from "node:path";
+import { spawn } from "node:child_process";
+import { fileURLToPath } from "node:url";
+
+const currentFilePath = fileURLToPath(import.meta.url);
+const repoRoot = path.resolve(path.dirname(currentFilePath), "..");
+
+export const DEFAULT_MANIFEST_PATH = "scripts/lib/workflow-reliability-release-check.json";
+export const KNOWN_PACKAGES = new Map([
+  ["@fusion/core", "packages/core"],
+  ["@fusion/dashboard", "packages/dashboard"],
+  ["@fusion/engine", "packages/engine"],
+]);
+export const DEFAULT_COMMAND_TIMEOUT_MS = 300_000;
+
+/**
+ * FNXC:CustomWorkflowReliability 2026-06-19-00:00:
+ * FN-6694 makes the custom-workflow reliability release checklist executable for QA without adding it to the merge gate. Keep this harness on-demand, targeted to manifest-listed Vitest seams only, and fail closed when a manifest seam path is missing so release evidence cannot silently drift.
+ */
+export function loadManifest(manifestPath = DEFAULT_MANIFEST_PATH, { rootDir = repoRoot } = {}) {
+  const absolutePath = path.isAbsolute(manifestPath) ? manifestPath : path.join(rootDir, manifestPath);
+  return JSON.parse(readFileSync(absolutePath, "utf8"));
+}
+
+function normalizeManual(manifest) {
+  const manualEntries = Array.isArray(manifest?.manual) ? manifest.manual : [];
+  return new Map(manualEntries.map((entry) => [entry?.id, entry]));
+}
+
+function validateSeam(seam, item, index, seamIndex, { repoRoot: rootDir, existsSync: fileExists }) {
+  const errors = [];
+  const label = `checklist[${index}] ${item?.id ?? "<missing-id>"} seams[${seamIndex}]`;
+  if (!seam || typeof seam !== "object") {
+    errors.push(`${label} must be an object`);
+    return errors;
+  }
+  if (!KNOWN_PACKAGES.has(seam.package)) {
+    errors.push(`${label} references unknown package ${JSON.stringify(seam.package)}`);
+  }
+  if (!seam.file || typeof seam.file !== "string") {
+    errors.push(`${label} must include a repo-relative file`);
+  } else {
+    const normalizedFile = path.normalize(seam.file);
+    if (path.isAbsolute(seam.file) || normalizedFile.startsWith("..") || normalizedFile.includes(`${path.sep}..${path.sep}`)) {
+      errors.push(`${label} file must stay within the repository: ${seam.file}`);
+    } else if (!fileExists(path.join(rootDir, seam.file))) {
+      errors.push(`${label} file does not exist: ${seam.file}`);
+    }
+  }
+  return errors;
+}
+
+export function validateManifest(manifest, { repoRoot: rootDir = repoRoot, existsSync: fileExists = existsSync } = {}) {
+  const errors = [];
+  if (!manifest || typeof manifest !== "object") {
+    return { ok: false, errors: ["manifest must be an object"] };
+  }
+  if (manifest.version !== 1) errors.push("manifest.version must be 1");
+  if (!Array.isArray(manifest.checklist)) errors.push("manifest.checklist must be an array");
+
+  const manualById = normalizeManual(manifest);
+  const checklist = Array.isArray(manifest.checklist) ? manifest.checklist : [];
+  const seenIds = new Set();
+  checklist.forEach((item, index) => {
+    const label = `checklist[${index}]`;
+    if (!item || typeof item !== "object") {
+      errors.push(`${label} must be an object`);
+      return;
+    }
+    if (!item.id || typeof item.id !== "string") {
+      errors.push(`${label} must include an id`);
+    } else if (seenIds.has(item.id)) {
+      errors.push(`${label} duplicates id ${item.id}`);
+    } else {
+      seenIds.add(item.id);
+    }
+    if (!item.title || typeof item.title !== "string") errors.push(`${label} ${item.id ?? "<missing-id>"} must include a title`);
+    if (!item.journey || typeof item.journey !== "string") errors.push(`${label} ${item.id ?? "<missing-id>"} must include a journey`);
+
+    const seams = Array.isArray(item.seams) ? item.seams : [];
+    const manual = manualById.get(item.id);
+    const manualReason = typeof manual?.automationDeferredReason === "string" ? manual.automationDeferredReason.trim() : "";
+    if (seams.length === 0 && manualReason.length === 0) {
+      errors.push(`${label} ${item.id ?? "<missing-id>"} must have at least one seam or a manual automationDeferredReason`);
+    }
+    seams.forEach((seam, seamIndex) => {
+      errors.push(...validateSeam(seam, item, index, seamIndex, { repoRoot: rootDir, existsSync: fileExists }));
+    });
+  });
+
+  const manual = Array.isArray(manifest.manual) ? manifest.manual : [];
+  manual.forEach((entry, index) => {
+    if (!entry?.id || typeof entry.id !== "string") errors.push(`manual[${index}] must include an id`);
+    if (!entry?.title || typeof entry.title !== "string") errors.push(`manual[${index}] ${entry?.id ?? "<missing-id>"} must include a title`);
+    if (!entry?.automationDeferredReason || typeof entry.automationDeferredReason !== "string" || entry.automationDeferredReason.trim().length === 0) {
+      errors.push(`manual[${index}] ${entry?.id ?? "<missing-id>"} must include a non-empty automationDeferredReason`);
+    }
+    if (entry?.id && !seenIds.has(entry.id)) errors.push(`manual[${index}] references unknown checklist id ${entry.id}`);
+  });
+
+  return { ok: errors.length === 0, errors };
+}
+
+function distinctSeams(manifest) {
+  const seen = new Set();
+  const seams = [];
+  for (const item of manifest.checklist ?? []) {
+    for (const seam of item.seams ?? []) {
+      const key = `${seam.package}\u0000${seam.file}`;
+      if (seen.has(key)) continue;
+      seen.add(key);
+      seams.push({ package: seam.package, file: seam.file });
+    }
+  }
+  return seams;
+}
+
+export function planCommands(manifest) {
+  const byPackage = new Map();
+  for (const seam of distinctSeams(manifest)) {
+    if (!byPackage.has(seam.package)) byPackage.set(seam.package, []);
+    byPackage.get(seam.package).push(seam.file);
+  }
+
+  return [...byPackage.entries()].map(([packageName, files]) => {
+    const packageRoot = KNOWN_PACKAGES.get(packageName);
+    const packageRelativeFiles = files.map((file) => path.relative(packageRoot, file));
+    return {
+      package: packageName,
+      files,
+      command: "pnpm",
+      args: ["--filter", packageName, "exec", "vitest", "run", ...packageRelativeFiles, "--silent=passed-only", "--reporter=dot"],
+    };
+  });
+}
+
+function runProcess(command, args, { cwd, timeoutMs, stdout = process.stdout, stderr = process.stderr } = {}) {
+  return new Promise((resolve) => {
+    const child = spawn(command, args, { cwd, shell: false, detached: false, stdio: ["ignore", "pipe", "pipe"] });
+    let stdoutText = "";
+    let stderrText = "";
+    let timedOut = false;
+    const timer = setTimeout(() => {
+      timedOut = true;
+      child.kill("SIGTERM");
+    }, timeoutMs);
+
+    child.stdout.on("data", (chunk) => {
+      const text = String(chunk);
+      stdoutText += text;
+      stdout?.write?.(text);
+    });
+    child.stderr.on("data", (chunk) => {
+      const text = String(chunk);
+      stderrText += text;
+      stderr?.write?.(text);
+    });
+    child.on("error", (error) => {
+      clearTimeout(timer);
+      resolve({ exitCode: 1, timedOut, stdout: stdoutText, stderr: `${stderrText}${error.message}\n` });
+    });
+    child.on("close", (exitCode, signal) => {
+      clearTimeout(timer);
+      resolve({ exitCode: exitCode ?? 1, signal, timedOut, stdout: stdoutText, stderr: stderrText });
+    });
+  });
+}
+
+export async function runReleaseCheck(manifest, { rootDir = repoRoot, timeoutMs = DEFAULT_COMMAND_TIMEOUT_MS, stdout = process.stdout, stderr = process.stderr } = {}) {
+  const commands = planCommands(manifest);
+  const seamResults = new Map();
+  for (const planned of commands) {
+    stdout?.write?.(`Running: ${planned.command} ${planned.args.join(" ")}\n`);
+    const result = await runProcess(planned.command, planned.args, { cwd: rootDir, timeoutMs, stdout, stderr });
+    for (const file of planned.files) {
+      seamResults.set(file, {
+        package: planned.package,
+        file,
+        status: result.exitCode === 0 && !result.timedOut ? "PASS" : "FAIL",
+        exitCode: result.exitCode,
+        timedOut: result.timedOut,
+        command: `${planned.command} ${planned.args.join(" ")}`,
+      });
+    }
+  }
+
+  return (manifest.checklist ?? []).map((item) => {
+    const seams = (item.seams ?? []).map((seam) => seamResults.get(seam.file) ?? { ...seam, status: "FAIL", exitCode: null, timedOut: false, command: null });
+    const manual = (manifest.manual ?? []).find((entry) => entry.id === item.id) ?? null;
+    return { id: item.id, title: item.title, journey: item.journey, seams, manual };
+  });
+}
+
+export function summarize(results) {
+  const items = results.map((result) => {
+    const seamStatuses = (result.seams ?? []).map((seam) => seam.status);
+    const hasFailure = seamStatuses.includes("FAIL");
+    const hasPass = seamStatuses.includes("PASS");
+    const manualReason = typeof result.manual?.automationDeferredReason === "string" ? result.manual.automationDeferredReason.trim() : "";
+    const status = hasFailure ? "FAIL" : hasPass ? "PASS" : manualReason ? "MANUAL" : "FAIL";
+    return { ...result, status };
+  });
+  const counts = {
+    pass: items.filter((item) => item.status === "PASS").length,
+    fail: items.filter((item) => item.status === "FAIL").length,
+    manual: items.filter((item) => item.status === "MANUAL").length,
+    total: items.length,
+  };
+  return { ok: counts.fail === 0, counts, items };
+}
+
+export function renderReport(summary, { json = false } = {}) {
+  if (json) return `${JSON.stringify(summary, null, 2)}\n`;
+  const lines = [
+    "Custom workflow reliability release-check",
+    `Overall: ${summary.ok ? "PASS" : "FAIL"} (${summary.counts.pass} passed, ${summary.counts.fail} failed, ${summary.counts.manual} manual, ${summary.counts.total} total)`,
+    "",
+  ];
+  for (const item of summary.items) {
+    lines.push(`${item.status}: ${item.id} — ${item.title}`);
+    for (const seam of item.seams ?? []) {
+      lines.push(`  - ${seam.status}: ${seam.file} (${seam.package})`);
+    }
+    if (item.manual?.automationDeferredReason) {
+      lines.push(`  - MANUAL: ${item.manual.automationDeferredReason}`);
+    }
+  }
+  return `${lines.join("\n")}\n`;
+}
+
+function parseArgs(argv) {
+  const args = { dryRun: false, json: false, manifestPath: DEFAULT_MANIFEST_PATH, timeoutMs: DEFAULT_COMMAND_TIMEOUT_MS };
+  for (let index = 0; index < argv.length; index += 1) {
+    const arg = argv[index];
+    if (arg === "--dry-run") args.dryRun = true;
+    else if (arg === "--json") args.json = true;
+    else if (arg === "--manifest") args.manifestPath = argv[++index];
+    else if (arg === "--timeout-ms") args.timeoutMs = Number(argv[++index]);
+    else if (arg === "--help" || arg === "-h") args.help = true;
+    else throw new Error(`Unknown argument: ${arg}`);
+  }
+  if (!Number.isFinite(args.timeoutMs) || args.timeoutMs <= 0) throw new Error("--timeout-ms must be a positive number");
+  return args;
+}
+
+function renderDryRun(manifest) {
+  const commands = planCommands(manifest);
+  const results = (manifest.checklist ?? []).map((item) => ({
+    id: item.id,
+    title: item.title,
+    journey: item.journey,
+    seams: (item.seams ?? []).map((seam) => ({ ...seam, status: "PASS", exitCode: 0, timedOut: false, command: null })),
+    manual: (manifest.manual ?? []).find((entry) => entry.id === item.id) ?? null,
+  }));
+  return { commands, summary: summarize(results) };
+}
+
+export async function main(argv = process.argv.slice(2), { rootDir = repoRoot, stdout = process.stdout, stderr = process.stderr } = {}) {
+  let args;
+  try {
+    args = parseArgs(argv);
+  } catch (error) {
+    stderr.write(`${error.message}\n`);
+    return 1;
+  }
+  if (args.help) {
+    stdout.write("Usage: node scripts/workflow-reliability-release-check.mjs [--dry-run] [--json] [--manifest <path>] [--timeout-ms <ms>]\n");
+    return 0;
+  }
+
+  let manifest;
+  try {
+    manifest = loadManifest(args.manifestPath, { rootDir });
+  } catch (error) {
+    const summary = { ok: false, counts: { pass: 0, fail: 1, manual: 0, total: 1 }, items: [{ id: "manifest", title: "Manifest load", status: "FAIL", seams: [], error: error.message }] };
+    stdout.write(renderReport(summary, { json: args.json }));
+    return 1;
+  }
+
+  const validation = validateManifest(manifest, { repoRoot: rootDir, existsSync });
+  if (!validation.ok) {
+    const summary = { ok: false, counts: { pass: 0, fail: validation.errors.length, manual: 0, total: validation.errors.length }, items: validation.errors.map((error, index) => ({ id: `manifest-${index + 1}`, title: error, status: "FAIL", seams: [] })) };
+    stdout.write(renderReport(summary, { json: args.json }));
+    return 1;
+  }
+
+  if (args.dryRun) {
+    const dryRun = renderDryRun(manifest);
+    if (args.json) {
+      stdout.write(`${JSON.stringify({ ok: dryRun.summary.ok, dryRun: true, commands: dryRun.commands, summary: dryRun.summary }, null, 2)}\n`);
+    } else {
+      stdout.write("Dry run: manifest is valid. Planned commands:\n");
+      for (const command of dryRun.commands) stdout.write(`- ${command.command} ${command.args.join(" ")}\n`);
+      stdout.write("\n");
+      stdout.write(renderReport(dryRun.summary));
+    }
+    return dryRun.summary.ok ? 0 : 1;
+  }
+
+  const results = await runReleaseCheck(manifest, { rootDir, timeoutMs: args.timeoutMs, stdout: args.json ? { write: () => {} } : stdout, stderr });
+  const summary = summarize(results);
+  stdout.write(renderReport(summary, { json: args.json }));
+  return summary.ok ? 0 : 1;
+}
+
+if (import.meta.url === `file://${process.argv[1]}`) {
+  const exitCode = await main();
+  process.exitCode = exitCode;
+}