feat: add gate check for CLI dashboard mock completeness — prevents recurring full-suite barrel-export drift (#2035)

## Summary

**Structural fix** for the recurring full-suite failure pattern where a
new `@fusion/dashboard` barrel export is imported by CLI source code but
missing from the hardcoded `vi.mock("@fusion/dashboard")` factory in CLI
tests.

## What's new

### Gate check script:
`scripts/check-cli-dashboard-mock-completeness.mjs`
Added to the merge gate (`pnpm test:gate`). Statically validates that
every hardcoded `vi.mock("@fusion/dashboard")` factory in CLI tests
includes all `@fusion/dashboard` exports that the corresponding source
files import.

- Pure static analysis (regex + depth-aware brace tracking) — no module
evaluation, <0.1s
- Handles named imports (`import { foo } from "@fusion/dashboard"`) AND
namespace imports (`import * as dashboard from "@fusion/dashboard"` →
scans `dashboard.X` usages)
- Filters against the real barrel exports to avoid false positives from
typos
- Resolves test→source mapping by parsing static/dynamic imports in the
test file (not just naming convention)

**Result:** the next time someone adds `export { newFunc } from
"./mod.js"` to `dashboard/src/index.ts` and `cli/src/commands/daemon.ts`
imports it, the gate catches the missing mock before merge instead of
the full-suite failing on main.

### Completed all 9 incomplete CLI dashboard mocks
Added the missing exports identified by the check:

| File | Missing exports added |
|---|---|
| `daemon.test.ts` | `registerGithubTrackingHook` |
| `serve.test.ts` | `registerGithubTrackingHook` |
| `dashboard.test.ts` | `AttachTicketStore`, `CliInputAttributionLog`,
`CliConfirmAdvanceRegistry`, `CliRelaunchRegistry`,
`registerGithubTrackingHook` |
| `task.test.ts` | `registerGithubTrackingHook`, `GitLabClient`,
`resolveGitlabAuth`, `buildGitLabTaskProvenance`,
`isGitLabAlreadyImported`, `buildGitLabTaskDescription` |
| `extension-*.test.ts` (×4) | `GitLabClient`, `resolveGitlabAuth`,
`buildGitLabTaskProvenance`, `isGitLabAlreadyImported`,
`buildGitLabTaskDescription` |
| `task-command-github-import-tracking.test.ts` | Same GitLab exports |

These were latent issues — the mocks were incomplete but tests passed
because the missing exports weren't called during test execution. Any
test change that exercises those code paths would have broken.

## Why not `importActual` spread?
Tried converting daemon.test.ts to `vi.mock("@fusion/dashboard", async
(importOriginal) => { ... })` — fails because the barrel's `export *
from "./plugins/index.js"` transitively imports
`@agentclientprotocol/sdk` which isn't available at test evaluation
time. The static check approach avoids this entirely.

## Verification
- `pnpm test:gate`: exit 0 (includes new check)
- `pnpm lint`: exit 0
- CLI tests: daemon 21/21, serve 58/58, dashboard 91/91, task 149/149 ✅
- Gate script: `✅ CLI dashboard mock completeness: all hardcoded mocks
cover source imports.`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Tests**
- Added automated validation to ensure CLI test mocks remain aligned
with available dashboard functionality.
- Updated test coverage setup so GitHub, GitLab, daemon, dashboard,
server, and task scenarios use complete dashboard mocks.
- Test verification now reports missing mocked functionality and blocks
the release gate when inconsistencies are detected.

- **Chores**
- Improved reliability and maintainability of automated verification for
CLI and dashboard integrations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
gsxdsm
2026-07-12 23:09:50 -07:00
committed by GitHub
parent 2aefaad319
commit c8999369c3
11 changed files with 288 additions and 2 deletions

View File

@@ -19,7 +19,7 @@
"check:line-count": "node scripts/check-file-line-count.mjs",
"check:changesets": "node scripts/check-changeset-format.mjs",
"check:quarantine-ledger": "node scripts/check-quarantine-ledger.mjs",
"test:gate": "node scripts/check-no-nohup.mjs && node scripts/check-no-kill-4040.mjs && node scripts/check-no-test-timeout-appeasement.mjs && node scripts/check-changeset-format.mjs && pnpm --filter @fusion/engine test:core && pnpm --filter @runfusion/fusion test:ci-shape",
"test:gate": "node scripts/check-no-nohup.mjs && node scripts/check-no-kill-4040.mjs && node scripts/check-no-test-timeout-appeasement.mjs && node scripts/check-changeset-format.mjs && node scripts/check-cli-dashboard-mock-completeness.mjs && pnpm --filter @fusion/engine test:core && pnpm --filter @runfusion/fusion test:ci-shape",
"smoke:boot": "node scripts/boot-smoke.mjs",
"local": "node scripts/start-local.mjs",
"dev": "node scripts/dev-with-memory.mjs",

View File

@@ -53,6 +53,12 @@ vi.mock("@fusion/core", () => ({
vi.mock("@fusion/dashboard", () => ({
registerGithubTrackingHook: vi.fn(),
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
GitLabClient: vi.fn(),
resolveGitlabAuth: vi.fn(() => ({})),
buildGitLabTaskProvenance: vi.fn(() => ({})),
isGitLabAlreadyImported: vi.fn(),
buildGitLabTaskDescription: vi.fn(),
}));
vi.mock("@fusion/engine", () => ({

View File

@@ -9,7 +9,15 @@ const approvalFindLatestByDedupeKeyMock = vi.hoisted(() => vi.fn());
const recordRunAuditEventMock = vi.hoisted(() => vi.fn());
const assertNoSecretPlaintextMock = vi.hoisted(() => vi.fn());
vi.mock("@fusion/dashboard", () => ({ registerGithubTrackingHook: vi.fn() }));
vi.mock("@fusion/dashboard", () => ({
registerGithubTrackingHook: vi.fn(),
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
GitLabClient: vi.fn(),
resolveGitlabAuth: vi.fn(() => ({})),
buildGitLabTaskProvenance: vi.fn(() => ({})),
isGitLabAlreadyImported: vi.fn(),
buildGitLabTaskDescription: vi.fn(),
}));
vi.mock("@fusion/engine", () => ({
...workflowAuthoringEngineMock,
createFnAgent: vi.fn(),

View File

@@ -19,6 +19,12 @@ const registerGithubTrackingHookMock = vi.hoisted(() => vi.fn(() => {
vi.mock("@fusion/dashboard", () => ({
registerGithubTrackingHook: registerGithubTrackingHookMock,
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
GitLabClient: vi.fn(),
resolveGitlabAuth: vi.fn(() => ({})),
buildGitLabTaskProvenance: vi.fn(() => ({})),
isGitLabAlreadyImported: vi.fn(),
buildGitLabTaskDescription: vi.fn(),
}));
vi.mock("@fusion/core/gh-cli", () => ({

View File

@@ -5,6 +5,12 @@ const fetchWebContentMock = vi.hoisted(() => vi.fn());
vi.mock("@fusion/dashboard", () => ({
registerGithubTrackingHook: vi.fn(),
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
GitLabClient: vi.fn(),
resolveGitlabAuth: vi.fn(() => ({})),
buildGitLabTaskProvenance: vi.fn(() => ({})),
isGitLabAlreadyImported: vi.fn(),
buildGitLabTaskDescription: vi.fn(),
}));
vi.mock("@fusion/engine", () => ({

View File

@@ -42,6 +42,12 @@ vi.mock("../project-context.js", () => ({
vi.mock("@fusion/dashboard", () => ({
registerGithubTrackingHook: vi.fn(),
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
GitLabClient: vi.fn(),
resolveGitlabAuth: vi.fn(() => ({})),
buildGitLabTaskProvenance: vi.fn(() => ({})),
isGitLabAlreadyImported: vi.fn(),
buildGitLabTaskDescription: vi.fn(),
}));
vi.mock("@fusion/engine", () => ({

View File

@@ -576,6 +576,8 @@ resolveCliPackageVersionInfo: vi.fn(() => ({ version: "0.0.0-test", isUnresolved
getProjectSettingsPath: vi.fn().mockReturnValue("/tmp/project/.fusion/settings.json"),
loadTlsCredentialsFromEnv: vi.fn().mockReturnValue(undefined),
refreshAllCustomProviderModels: mocks.refreshAllCustomProviderModels,
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
registerGithubTrackingHook: vi.fn(),
}));
vi.mock("@fusion/engine", async (importOriginal) => {

View File

@@ -425,6 +425,12 @@ resolveCliPackageVersionInfo: vi.fn(() => ({ version: "0.0.0-test", isUnresolved
loadTlsCredentialsFromEnv: vi.fn().mockReturnValue(undefined),
refreshAllCustomProviderModels: mockRefreshAllCustomProviderModels,
stopAllDevServers: vi.fn().mockResolvedValue(undefined),
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
AttachTicketStore: vi.fn(),
CliInputAttributionLog: vi.fn(),
CliConfirmAdvanceRegistry: vi.fn(),
CliRelaunchRegistry: vi.fn(),
registerGithubTrackingHook: vi.fn(),
}));
// ── Mock node:readline ──────────────────────────────────────────────

View File

@@ -636,6 +636,8 @@ resolveCliPackageVersionInfo: vi.fn(() => ({ version: "0.0.0-test", isUnresolved
getProjectSettingsPath: vi.fn().mockReturnValue("/tmp/project/.fusion/settings.json"),
loadTlsCredentialsFromEnv: vi.fn().mockReturnValue(undefined),
refreshAllCustomProviderModels: mocks.refreshAllCustomProviderModels,
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
registerGithubTrackingHook: vi.fn(),
}));
vi.mock("@fusion/engine", async (importOriginal) => {

View File

@@ -113,6 +113,13 @@ vi.mock("@fusion/dashboard", () => ({
}),
generatePrMetadata: vi.fn(),
loadTlsCredentialsFromEnv: vi.fn().mockReturnValue(undefined),
// FNXC:CliTests 2026-07-13-09:40: Missing dashboard barrel exports added for mock completeness (scripts/check-cli-dashboard-mock-completeness.mjs gate).
registerGithubTrackingHook: vi.fn(),
GitLabClient: vi.fn(),
resolveGitlabAuth: vi.fn(() => ({})),
buildGitLabTaskProvenance: vi.fn(() => ({})),
isGitLabAlreadyImported: vi.fn(),
buildGitLabTaskDescription: vi.fn(),
}));
vi.mock("@fusion/dashboard/planning", () => ({

View File

@@ -0,0 +1,237 @@
/*
* FNXC:TestInfrastructure 2026-07-13-09:30:
* Static gate check that prevents the recurring full-suite failure pattern where
* a new export added to the @fusion/dashboard barrel is imported by CLI source
* code but missing from the hardcoded vi.mock("@fusion/dashboard") factory in
* the corresponding CLI test file.
*
* This runs as part of the merge gate (pnpm test:gate) so drift is caught
* before merge, not after full-suite fails on main.
*
* The check is purely static (regex-based, no module evaluation) and fast (<1s).
*/
import { readFileSync, readdirSync, statSync, existsSync } from "node:fs";
import { join, dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const __dirname = dirname(fileURLToPath(import.meta.url));
const root = resolve(__dirname, "..");
const cliSrc = join(root, "packages/cli/src");
// ── 1. Extract value exports from the dashboard barrel ──────────────────────
const barrelPath = join(root, "packages/dashboard/src/index.ts");
const barrelSrc = readFileSync(barrelPath, "utf8");
/**
* Extract named exports from the barrel, excluding type-only exports.
* Handles: export { foo, type Bar, baz as qux } from "./mod.js";
*/
function extractBarrelExports(src) {
const exports = new Set();
const namedRe = /export\s*\{([^}]+)\}\s*from\s*"[^"]+"/g;
let m;
while ((m = namedRe.exec(src)) !== null) {
for (let raw of m[1].split(",")) {
raw = raw.trim();
if (!raw || raw.startsWith("type ")) continue;
const aliased = raw.split(/\s+as\s+/);
const name = (aliased[aliased.length - 1] || raw).trim();
if (name && /^[A-Za-z_]/.test(name)) exports.add(name);
}
}
return exports;
}
const barrelExports = extractBarrelExports(barrelSrc);
// ── 2. Extract @fusion/dashboard usage from a source file ───────────────────
/**
* Extract dashboard member usage from a source file.
* Handles both named imports and namespace imports.
* Returns a Set of member names.
*/
function extractDashboardUsage(filePath) {
let src;
try { src = readFileSync(filePath, "utf8"); } catch { return new Set(); }
const used = new Set();
// Named imports: import { A, type B, C as D } from "@fusion/dashboard"
const namedRe = /import\s*\{([^}]+)\}\s*from\s*"@fusion\/dashboard"/g;
let m;
while ((m = namedRe.exec(src)) !== null) {
for (let raw of m[1].split(",")) {
raw = raw.trim();
if (!raw || raw.startsWith("type ")) continue;
const aliased = raw.split(/\s+as\s+/);
const name = (aliased[0] || raw).trim();
if (name && /^[A-Za-z_]/.test(name)) used.add(name);
}
}
// Namespace imports: import * as X from "@fusion/dashboard"
// Then find all X.member usages.
const nsRe = /import\s*\*\s*as\s+(\w+)\s*from\s*"@fusion\/dashboard"/g;
while ((m = nsRe.exec(src)) !== null) {
const ns = m[1];
const memberRe = new RegExp(`\\b${ns}\\.(\\w+)`, "g");
let mm;
while ((mm = memberRe.exec(src)) !== null) {
used.add(mm[1]);
}
}
return used;
}
// ── 3. Resolve source files from a test file ────────────────────────────────
/**
* Find source files a test covers by:
* 1. Parsing static/dynamic imports in the test file
* 2. Convention: __tests__/foo.test.ts → ../foo.ts
*/
function resolveSourceFiles(testPath) {
const sources = new Set();
const testDir = dirname(testPath);
let testSrc;
try { testSrc = readFileSync(testPath, "utf8"); } catch { return sources; }
// Static imports: import { ... } from "../foo.js" or "../commands/bar.js"
const staticRe = /import\s+(?:type\s+)?[\w{},\s*]*\s*from\s*"(\.\.?\/[^"]+\.js)"/g;
let m;
while ((m = staticRe.exec(testSrc)) !== null) {
const resolved = resolve(testDir, m[1].replace(/\.js$/, ".ts"));
if (existsSync(resolved)) sources.add(resolved);
}
// Dynamic imports: await import("../foo.js") or import("../commands/bar.js")
const dynRe = /import\(\s*"(\.\.?\/[^"]+\.js)"\s*\)/g;
while ((m = dynRe.exec(testSrc)) !== null) {
const resolved = resolve(testDir, m[1].replace(/\.js$/, ".ts"));
if (existsSync(resolved)) sources.add(resolved);
}
// Convention fallback: __tests__/foo.test.ts → ../foo.ts
const noTests = testPath.replace(/__tests\//, "");
const convPath = noTests.replace(/\.test\.ts$/, ".ts");
if (existsSync(convPath)) sources.add(convPath);
// bin.test.ts special case
if (testPath.endsWith("__tests__/bin.test.ts")) {
const binPath = join(cliSrc, "bin.ts");
if (existsSync(binPath)) sources.add(binPath);
}
return sources;
}
// ── 4. Extract mock keys from a hardcoded vi.mock factory ───────────────────
/**
* Extract mock keys from vi.mock("@fusion/dashboard", () => ({ ... })).
* Returns null if mock uses importOriginal/importActual (auto-spread, safe).
*/
function extractMockKeys(testSrc) {
if (/vi\.mock\(\s*"@fusion\/dashboard"[^)]*importOriginal/.test(testSrc) ||
/vi\.mock\(\s*"@fusion\/dashboard"[^)]*importActual/.test(testSrc)) {
return null;
}
// Find the start of the mock factory object: () => ({
const startRe = /vi\.mock\(\s*"@fusion\/dashboard"\s*,\s*\([^)]*\)\s*=>\s*\(\s*\{/;
const startMatch = startRe.exec(testSrc);
if (!startMatch) return null;
// Depth-aware extraction: track { } depth to find the matching close
const bodyStart = startMatch.index + startMatch[0].length;
let depth = 1;
let i = bodyStart;
while (i < testSrc.length && depth > 0) {
const ch = testSrc[i];
if (ch === "{") depth++;
else if (ch === "}") depth--;
i++;
}
const body = testSrc.slice(bodyStart, i - 1);
// Extract property keys from the mock body
const keys = new Set();
const keyRe = /(?:^|\n)\s*([A-Za-z_$][\w$]*)\s*(?::)/g;
let km;
while ((km = keyRe.exec(body)) !== null) {
keys.add(km[1]);
}
return keys;
}
// ── 5. Run the check ─────────────────────────────────────────────────────────
/**
* Recursively collect .ts files under a directory.
*/
function collectTs(dir) {
let out = [];
try {
for (const entry of readdirSync(dir)) {
const full = join(dir, entry);
const st = statSync(full);
if (st.isDirectory()) out = out.concat(collectTs(full));
else if (entry.endsWith(".ts")) out.push(full);
}
} catch { /* dir may not exist */ }
return out;
}
const testFiles = collectTs(join(cliSrc, "__tests__"))
.concat(collectTs(join(cliSrc, "commands", "__tests__")))
.concat(collectTs(join(cliSrc, "plugins", "__tests__")))
.filter(f => f.endsWith(".test.ts"));
const errors = [];
for (const testFile of testFiles) {
const testSrc = readFileSync(testFile, "utf8");
if (!testSrc.includes('vi.mock("@fusion/dashboard"')) continue;
const mockKeys = extractMockKeys(testSrc);
if (mockKeys === null) continue;
// Collect required dashboard exports from all source files this test covers
const sourcePaths = resolveSourceFiles(testFile);
const requiredExports = new Set();
for (const srcPath of sourcePaths) {
const used = extractDashboardUsage(srcPath);
for (const e of used) {
// Only flag exports that actually exist in the dashboard barrel
// (filters out namespace typos and non-export members)
if (barrelExports.has(e)) requiredExports.add(e);
}
}
// Check: every required export must be in mock keys
const missing = [...requiredExports].filter(e => !mockKeys.has(e));
if (missing.length > 0) {
const rel = testFile.replace(root + "/", "");
errors.push(
` ${rel}\n missing: ${missing.map(m => `"${m}"`).join(", ")}\n` +
` (imported from @fusion/dashboard in source, absent from vi.mock factory)\n` +
` resolved sources: ${[...sourcePaths].map(s => s.replace(root + "/", "")).join(", ") || "(none found)"}`
);
}
}
if (errors.length > 0) {
console.error(
`\n❌ CLI dashboard mock completeness check failed (${errors.length} issue${errors.length > 1 ? "s" : ""}):\n`
);
for (const e of errors) console.error(e + "\n");
console.error(
`Fix: add the missing export(s) to each vi.mock("@fusion/dashboard") factory.`
);
process.exit(1);
} else {
console.log("✅ CLI dashboard mock completeness: all hardcoded mocks cover source imports.");
}