fix(FN-5048): direct agent + tool verification away from full workspace suite
Tighten the executor agent guidance and the fn_run_verification tool guidance so verification stays scoped to changed files instead of running the full workspace test suite, which (for a foundational-package edit) reverse-expands across the whole workspace and stalls the task. - agent-prompts.ts: remove the "during final integration" blanket permission to run workspace-wide suites; name the forbidden full-run commands explicitly. - run-verification-tool.ts: strengthen BOUNDED_VERIFICATION_GUIDANCE to forbid `pnpm test:full` / `pnpm verify:workspace` / whole-package runs as verification. Engine + core typecheck pass; run-verification-command and executor-review-verdicts prompt-assertion suites stay green (93 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -2,4 +2,4 @@
|
||||
"@runfusion/fusion": patch
|
||||
---
|
||||
|
||||
Keep Fusion verification progress moving by making targeted script tests honor file arguments, reaping verification subprocess groups after clean exits, and preventing the line-count audit from blocking `pnpm test`.
|
||||
Keep Fusion verification progress moving by making targeted script tests honor file arguments, reaping verification subprocess groups after clean exits, and preventing the line-count audit from blocking `pnpm test`. The changed-test runner now caps reverse-dependent fan-out so a foundational-package edit no longer expands into a whole-workspace run, and the executor/verification guidance now directs agents to scope verification to changed files rather than running the full workspace test suite.
|
||||
|
||||
@@ -295,7 +295,7 @@ For bug-class/bug-fix tasks, add and fill in the exact \`## Symptom Verification
|
||||
|
||||
### Step {N-1}: Testing & Verification
|
||||
|
||||
> ZERO failures allowed for checks required by this task's quality gates. Run impacted/package-scoped verification first; run workspace-wide suites only when the task or workflow explicitly requires them, or during final integration after impacted checks pass.
|
||||
> ZERO failures allowed for checks required by this task's quality gates. Run impacted/package-scoped verification first. Do NOT run the full workspace test suite (\`pnpm test:full\`, \`pnpm verify:workspace\`, or whole-package \`pnpm --filter <pkg> test\`) as routine or final-integration verification — a full run is allowed ONLY when the task or workflow explicitly requires it.
|
||||
> If keeping lint/tests/build/typecheck green requires edits outside the initial File Scope, make those fixes as part of this task.
|
||||
|
||||
- [ ] Run lint check (\`pnpm lint\`)
|
||||
@@ -363,7 +363,7 @@ If this task REMOVES existing functionality (deleting modules, settings, API end
|
||||
- For bug fixes and UI-affordance add/remove tasks, populate \`## Surface Enumeration\` with this checklist from \`docs/testing.md\`: providers/bridges/execution paths; desktop + mobile breakpoints/platforms; empty/undefined/duplicate/populated data states; shared hooks/components/modules/helpers; every component that renders the affordance; leftover shells after removal.
|
||||
- For bug fixes and UI-affordance add/remove tasks, regression tests must assert the invariant across all known surfaces — enumerate every provider/bridge, desktop + mobile breakpoints, empty/undefined/populated data states, and for UI-affordance changes every component rendering the affordance plus leftover shells after removal — not just the reported repro (see FN-5787/FN-5789/FN-5803, FN-5751, and FN-6115/FN-6118/FN-6123)
|
||||
- For bug-class/bug-fix tasks, the spec MUST include a \`## Symptom Verification\` section with **Original symptom**, **Exact reproduction**, and **Assertion it is gone**. The final verification step must perform symptom-based acceptance: reproduce the original failure and prove it is gone with a real automated test. Green build/tests alone are insufficient. Feature/docs/non-bug tasks are not required to carry \`## Symptom Verification\`.
|
||||
- Include targeted tests in implementation steps and full quality-gate runs in final verification
|
||||
- Include targeted tests in implementation steps and bounded/changed-scoped quality-gate runs in final verification; never run the full workspace test suite unless the task explicitly requires it
|
||||
|
||||
## Duplicate check
|
||||
Before writing a spec, call \`fn_task_list\` to find existing active tasks, then call \`fn_task_search\` with 2-4 distinct keyword phrases from the task title and description (for example file paths, error symptoms, and symbol names).
|
||||
@@ -525,7 +525,7 @@ For bug-class/bug-fix tasks, add and fill in the exact \`## Symptom Verification
|
||||
|
||||
### Step {N-1}: Testing & Verification
|
||||
|
||||
> ZERO failures allowed for checks required by this task's quality gates. Run impacted/package-scoped verification first; run workspace-wide suites only when the task or workflow explicitly requires them, or during final integration after impacted checks pass.
|
||||
> ZERO failures allowed for checks required by this task's quality gates. Run impacted/package-scoped verification first. Do NOT run the full workspace test suite (\`pnpm test:full\`, \`pnpm verify:workspace\`, or whole-package \`pnpm --filter <pkg> test\`) as routine or final-integration verification — a full run is allowed ONLY when the task or workflow explicitly requires it.
|
||||
> If keeping lint/tests/build/typecheck green requires edits outside the initial File Scope, make those fixes as part of this task.
|
||||
|
||||
- [ ] Run lint check (\`pnpm lint\`)
|
||||
|
||||
@@ -37,8 +37,13 @@ export const DEFAULT_TIMEOUT_PACKAGE_SEC = 300;
|
||||
export const DEFAULT_TIMEOUT_WORKSPACE_SEC = 900;
|
||||
export const MAX_TIMEOUT_SEC = 1800;
|
||||
|
||||
/*
|
||||
FNXC:Verification 2026-06-21-12:05:
|
||||
Verification must stay bounded — never run the full workspace test suite as the verification path.
|
||||
A foundational-package edit reverse-expands a full run across the whole workspace and stalls the task (see FN-5048 + the test-changed reverse-dependent blast cap); scope verification to the changed files/package instead.
|
||||
*/
|
||||
export const BOUNDED_VERIFICATION_GUIDANCE =
|
||||
"Prefer a bounded targeted command such as `pnpm --filter <pkg> exec vitest run src/path/to/test.ts --silent=passed-only --reporter=dot` before rerunning broader suites.";
|
||||
"Scope verification to the changed files: prefer a bounded targeted command such as `pnpm --filter <pkg> exec vitest run src/path/to/test.ts --silent=passed-only --reporter=dot`. Do NOT run the full workspace test suite (`pnpm test:full`, `pnpm verify:workspace`, or whole-package `pnpm --filter <pkg> test`) as verification.";
|
||||
export const MARATHON_SOFT_CAP_SEC = 120;
|
||||
|
||||
const packageDirCache = new Map<string, string | null>();
|
||||
@@ -315,6 +320,10 @@ function reapVerificationProcessGroup(supervised: SupervisedChild): void {
|
||||
* FNXC:Verification 2026-06-21-10:00:
|
||||
* Verification commands may spawn background test/dev children and then let the shell exit cleanly.
|
||||
* Reap the process group after normal close so fn_run_verification does not report completion while orphaned test workers keep later task progress stuck.
|
||||
*
|
||||
* FNXC:Verification 2026-06-21-10:26:
|
||||
* Apply this reap to every non-timeout close, including externally signal-terminated exits.
|
||||
* The supervisor kill path tolerates already-gone process groups, and the extra reap keeps all non-timeout exits from leaking background verification workers.
|
||||
*/
|
||||
killVerificationProcess(supervised, "SIGTERM");
|
||||
const forceKillTimer = setTimeout(() => {
|
||||
@@ -323,6 +332,10 @@ function reapVerificationProcessGroup(supervised: SupervisedChild): void {
|
||||
forceKillTimer.unref?.();
|
||||
}
|
||||
|
||||
export function __testOnlyReapVerificationProcessGroup(supervised: SupervisedChild): void {
|
||||
reapVerificationProcessGroup(supervised);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tool parameter schema
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user