docs(verification): instruct agents not to use allowFullSuite unless necessary
Strengthen the fn_run_verification allowFullSuite parameter description, add an AGENTS.md standing rule, and update docs/testing.md so agents default to a file-scoped verification command and reserve allowFullSuite for genuinely full runs with no targetable test set. allowFullSuite is the main way verification balloons past its budget; the thin merge gate is the cross-cutting safety net. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -131,6 +131,12 @@ pnpm verify:workspace # deep opt-in verification (lint -> test:full -> build);
|
||||
- Do not add new real-network calls, real polling loops, or mock-the-world shells when a narrower seam exists.
|
||||
- Use the testing taxonomy in `docs/testing.md` when deciding trim vs keep.
|
||||
|
||||
### Standing Rule: Scope Verification to Changed Files — Do Not Use `allowFullSuite`
|
||||
|
||||
- When verifying via `fn_run_verification`, **do not pass `allowFullSuite: true` unless absolutely necessary.** It is a last-resort escape hatch that runs a marathon command (root `pnpm test`, `pnpm test:full`, `verify:workspace`, whole-package tests, repeat loops) far in excess of what the change requires, and it is the main way verification balloons past its budget.
|
||||
- Default to a **file-scoped** command targeting only the tests affected by the diff, e.g. `pnpm --filter @fusion/<pkg> exec vitest run src/path/to/changed.test.ts --silent=passed-only --reporter=dot`. The marathon soft-cap exists to push you toward this.
|
||||
- `allowFullSuite: true` is justified only for a genuinely full run with no targetable test set (e.g. a cross-cutting infra change) — and then state the reason. The thin merge gate (`pnpm test:gate`) is the cross-cutting safety net, not per-task verification.
|
||||
|
||||
### Standing Rule: Fix the Invariant, Not the Repro (FN-5893)
|
||||
|
||||
- When fixing a bug, the regression test must assert the general invariant across ALL known surfaces — not only the single reported reproduction.
|
||||
|
||||
@@ -46,7 +46,7 @@ Custom workflow reliability release signoff has a dedicated on-demand lane: `pnp
|
||||
<!-- FNXC:iOSAcceptance 2026-06-18-17:25: Terminal acceptance gates that depend on real mobile Safari must use the credential-driven real-iOS surface runbook instead of treating desktop WebKit or jsdom as evidence. -->
|
||||
Terminal acceptance tasks that require real mobile Safari should use [`docs/ios-acceptance.md`](./ios-acceptance.md) for the `--check` run-vs-NO-OP probe, credential wiring, and physical/cloud real-iOS evidence workflow.
|
||||
|
||||
Agents running verification through `fn_run_verification` are bounded by default: project `verificationCommandTimeoutMs` when set, otherwise 300s for package scope and 900s for workspace scope, with an 1800s hard cap. Marathon invocations such as root `pnpm test`, `pnpm test:full`, `pnpm verify:workspace`, whole-package tests without file filters, and shell repeat loops are soft-capped unless the agent explicitly passes `allowFullSuite: true`; the escape hatch still emits progress heartbeats and respects the hard cap. Prefer targeted commands such as `pnpm --filter @fusion/<pkg> exec vitest run src/path/to/test.ts --silent=passed-only --reporter=dot` before opting into a full run.
|
||||
Agents running verification through `fn_run_verification` are bounded by default: project `verificationCommandTimeoutMs` when set, otherwise 300s for package scope and 900s for workspace scope, with an 1800s hard cap. Marathon invocations such as root `pnpm test`, `pnpm test:full`, `pnpm verify:workspace`, whole-package tests without file filters, and shell repeat loops are soft-capped unless the agent explicitly passes `allowFullSuite: true`; the escape hatch still emits progress heartbeats and respects the hard cap. **Do not pass `allowFullSuite: true` unless absolutely necessary** — it is the main way verification balloons past its budget. Default to a targeted, file-scoped command such as `pnpm --filter @fusion/<pkg> exec vitest run src/path/to/test.ts --silent=passed-only --reporter=dot`; reserve `allowFullSuite` for a genuinely full run with no targetable test set (state the reason), with the thin merge gate (`pnpm test:gate`) as the cross-cutting safety net.
|
||||
|
||||
## Fresh-worktree dist bootstrap
|
||||
|
||||
|
||||
@@ -367,7 +367,7 @@ export const runVerificationParams = Type.Object({
|
||||
allowFullSuite: Type.Optional(
|
||||
Type.Boolean({
|
||||
description:
|
||||
"Explicit opt-in for marathon verification commands such as pnpm test, pnpm test:full, verify:workspace, whole-package tests, or repeat loops. Default: false; still respects the hard timeout.",
|
||||
"DO NOT SET THIS unless absolutely necessary. It is a last-resort opt-in for marathon commands (`pnpm test`, `pnpm test:full`, `verify:workspace`, whole-package tests, repeat loops) that run far more than the change requires and make verification slow. Default false — keep it false. First scope verification to the changed files (e.g. `pnpm --filter <pkg> exec vitest run src/path/to/changed.test.ts --silent=passed-only --reporter=dot`); the soft cap exists to push you toward that. Only set true when a genuinely full run is unavoidable (e.g. a cross-cutting infra change with no targetable test set), and say why. Still respects the hard timeout.",
|
||||
}),
|
||||
),
|
||||
expectFailure: Type.Optional(
|
||||
|
||||
Reference in New Issue
Block a user