test(FN-000): harden local test suite
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Skipped Test Inventory
|
||||
|
||||
_Last audited: 2026-04-23 (FN-2346)_
|
||||
_Last audited: 2026-05-05_
|
||||
|
||||
This document tracks intentional skip usage in test suites so stale follow-up backlog items can be retired quickly.
|
||||
|
||||
@@ -9,18 +9,13 @@ This document tracks intentional skip usage in test suites so stale follow-up ba
|
||||
Audit commands:
|
||||
|
||||
```bash
|
||||
rg -n "\b(it|test|describe)\.skip\b|\bskipIf\b|createLoopbackIntegrationTest\(" packages --glob "**/*.{test,spec}.{ts,tsx}"
|
||||
rg -n "\?\s*it\s*:\s*it\.skip|detectLoopbackBinding" packages/dashboard/src --glob "**/*.{test,spec}.{ts,tsx}"
|
||||
rg -n "\b(it|test|describe)\.skip\b|\bskipIf\b|createLoopbackIntegrationTest\(" packages plugins scripts --glob "**/*.{test,spec}.{ts,tsx,mjs}"
|
||||
rg -n "\?\s*it\s*:\s*it\.skip|detectLoopbackBinding|const itPosix" packages plugins scripts --glob "**/*.{test,spec}.{ts,tsx,mjs}"
|
||||
```
|
||||
|
||||
Current results:
|
||||
|
||||
1. **Intentional cross-coverage alias**
|
||||
- `packages/engine/src/executor.test.ts`
|
||||
- `it.skip("step-session skill selection covered in step-session-executor.test.ts", ...)`
|
||||
- Rationale: dedicated coverage exists in `step-session-executor.test.ts`; this marker documents ownership.
|
||||
|
||||
2. **Environment-gated integration aliases (loopback gated)**
|
||||
1. **Environment-gated integration aliases (loopback gated)**
|
||||
- Canonical helper: `packages/dashboard/src/__tests__/loopback-integration-test.ts`
|
||||
- Helper consumers:
|
||||
- `packages/dashboard/src/server-static-assets.test.ts`
|
||||
@@ -30,18 +25,60 @@ Current results:
|
||||
- Rationale: these suites require real loopback binding support (`127.0.0.1`) and are intentionally environment-gated.
|
||||
- Auditability: when loopback binding is unavailable, skipped test names include a standardized reason and the suite scope label (`...; scope: <suite scope>`), making coverage gaps explicit in CI/test output.
|
||||
|
||||
3. **Build-output checks are now deterministic (no skip gate)**
|
||||
2. **CLI slow-lane agent export gate**
|
||||
- `packages/cli/src/commands/__tests__/agent-export.test.ts`
|
||||
- Pattern: `describe.skipIf(!SHOULD_RUN_SLOW_CLI)("agent-export", ...)`
|
||||
- Gate: `FUSION_TEST_SLOW_CLI=1` or `FUSION_TEST_SLOW_CLI=true`
|
||||
- Rationale: these tests perform real workspace and `AgentStore` round-trips and are kept out of the default CLI unit lane.
|
||||
- Replacement owner: the explicit slow lane (`pnpm --filter @runfusion/fusion test:slow-cli`) is responsible for running them when slow CLI coverage is requested.
|
||||
|
||||
3. **CLI pi extension integration gate**
|
||||
- `packages/cli/src/__tests__/extension-integration.test.ts`
|
||||
- Pattern: `describe.skipIf(!SHOULD_RUN_EXTENSION_INTEGRATION)("built fn pi extension integration", ...)`
|
||||
- Gate: `FUSION_TEST_EXTENSION_INTEGRATION=1` or `FUSION_TEST_EXTENSION_INTEGRATION=true`
|
||||
- Rationale: this built-extension suite requires compiled CLI artifacts, so it is run through the explicit local release lane instead of every unit run.
|
||||
- Replacement owner: `pnpm --filter @runfusion/fusion test:extension-integration`.
|
||||
|
||||
4. **CLI legacy pi extension gate**
|
||||
- `packages/cli/src/__tests__/extension.test.ts`
|
||||
- Pattern: `describe.skipIf(!SHOULD_RUN_LEGACY_EXTENSION_INTEGRATION)("fn pi extension (legacy exhaustive suite)", ...)`
|
||||
- Gate: `FUSION_TEST_LEGACY_EXTENSION_INTEGRATION=1` or `FUSION_TEST_LEGACY_EXTENSION_INTEGRATION=true`
|
||||
- Rationale: this exhaustive suite is intentionally excluded from default and release lanes while it remains useful only for historical debugging.
|
||||
- Replacement owner: `packages/cli/src/__tests__/extension-integration.test.ts` owns maintained built-extension coverage.
|
||||
|
||||
5. **CLI native binary build gate**
|
||||
- `packages/cli/src/__tests__/build-exe-cross.test.ts`
|
||||
- Pattern: four `describe.skipIf(!SHOULD_RUN_BUILD_EXE)(...)` suites for single-target, Windows-target, all-target, and default-target binary builds.
|
||||
- Gate: `FUSION_TEST_BUILD_EXE=1`, `FUSION_TEST_BUILD_EXE=true`, or `CI=true`
|
||||
- Rationale: cross-compiling native binaries is intentionally expensive and belongs in the binary/pre-release lane, not every local unit run.
|
||||
- Replacement owner: `packages/cli/src/__tests__/build-exe.test.ts` and bundle-output tests cover the default fast package contract; `build-exe-cross.test.ts` owns full cross-target coverage when the gate is enabled.
|
||||
|
||||
6. **POSIX-only shell syntax cases**
|
||||
- `packages/engine/src/__tests__/run-verification-command.test.ts`
|
||||
- `packages/engine/src/__tests__/verification-utils.test.ts`
|
||||
- Pattern: `const itPosix = onPosix ? it : it.skip`
|
||||
- Gate: skipped only on `process.platform === "win32"`
|
||||
- Rationale: a subset of tests uses POSIX shell syntax (`printf`, pipes, and shell quoting). The implementation still uses Node's portable `shell: true`; these specific fixtures are not portable to `cmd.exe`.
|
||||
- Replacement owner: platform-neutral cases in the same suite continue to run on Windows.
|
||||
|
||||
7. **Build-output checks are deterministic (no skip gate)**
|
||||
- `packages/cli/src/__tests__/bundle-output.test.ts`
|
||||
- `packages/dashboard/app/__tests__/build-output.test.ts`
|
||||
- Pattern: each suite builds required artifacts in `beforeAll` and then runs chunking/bundle assertions unconditionally.
|
||||
- Rationale: clean worktrees and CI environments should execute real output-contract assertions instead of silently skipping when `dist/` is absent.
|
||||
|
||||
8. **Skip gate string assertions are not skip markers**
|
||||
- `packages/cli/src/__tests__/ci-workflow.test.ts`
|
||||
- Pattern: assertions check that CI workflow text contains `describe.skipIf(...)` gate strings.
|
||||
- Rationale: this file does not skip tests itself; it verifies the workflow keeps the intentionally gated suites wired.
|
||||
|
||||
## Older Follow-up Reconciliation
|
||||
|
||||
Previously tracked actionable skip follow-ups are now resolved and should not be treated as open backlog:
|
||||
|
||||
- **FN-2085**: wildcard proxy POST body forwarding coverage is active.
|
||||
- **FN-2076 / FN-2106 / FN-2109**: NewAgentDialog and MissionInterviewModal rollback favorite-toggle regressions are active interaction tests.
|
||||
- The former engine `executor.test.ts` step-session alias skip is no longer present; ownership now lives in active engine tests.
|
||||
|
||||
Searches for those IDs in repository test code and docs now return no active TODO/skip markers tied to unresolved work.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user