FN-5893: require invariant-level bug regression coverage
Add invariant-first regression guidance to triage prompts and testing docs. - add an AGENTS standing rule to cover bug-fix invariants across all known surfaces - update standard, fast, and core triage prompt templates to require invariant-level regression tests - add triage regression tests that lock the new wording into standard, fast, and core prompts - link docs/testing guidance back to the new invariant-over-repro rule Files changed: AGENTS.md | 7 ++++++ docs/testing.md | 1 + packages/core/src/agent-prompts.ts | 3 ++- packages/engine/src/__tests__/triage.test.ts | 32 ++++++++++++++++++++++++++++ packages/engine/src/triage.ts | 6 ++++-- 5 files changed, 46 insertions(+), 3 deletions(-) Fusion-Task-Id: FN-5893 Fusion-Task-Lineage: 31c082c6-2a0a-4619-a13b-94dad97eef62
This commit is contained in:
@@ -9,9 +9,11 @@ import {
|
||||
computeUserCommentFingerprint,
|
||||
} from "../triage.js";
|
||||
import { join } from "node:path";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { mkdir, writeFile, rm, mkdtemp } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { planLog } from "../logger.js";
|
||||
|
||||
const { mockReviewStep, mockCreateFnAgent } = vi.hoisted(() => ({
|
||||
@@ -645,6 +647,36 @@ describe("TRIAGE_SYSTEM_PROMPT", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("FN-5893 invariant regression wording", () => {
|
||||
it("requires invariant-level regression coverage in standard, fast, and core triage prompts", () => {
|
||||
const corePromptSource = readFileSync(
|
||||
fileURLToPath(new URL("../../../core/src/agent-prompts.ts", import.meta.url)),
|
||||
"utf8",
|
||||
);
|
||||
|
||||
for (const prompt of [
|
||||
TRIAGE_SYSTEM_PROMPT,
|
||||
FAST_TRIAGE_SYSTEM_PROMPT,
|
||||
corePromptSource,
|
||||
]) {
|
||||
expect(prompt).toContain("invariant across all known surfaces");
|
||||
expect(prompt).toContain("provider/bridge");
|
||||
expect(prompt).toContain("desktop + mobile breakpoints");
|
||||
expect(prompt).toContain("empty/undefined/populated data states");
|
||||
expect(prompt).toContain("FN-5787/FN-5789/FN-5803");
|
||||
expect(prompt).toContain("FN-5751");
|
||||
}
|
||||
});
|
||||
|
||||
it("requires implementation-step testing guidance to enumerate invariant surfaces in standard and fast prompts", () => {
|
||||
for (const prompt of [TRIAGE_SYSTEM_PROMPT, FAST_TRIAGE_SYSTEM_PROMPT]) {
|
||||
expect(prompt).toContain(
|
||||
"Run targeted tests for changed files, asserting the invariant across all known surfaces",
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("fast-mode triage", () => {
|
||||
it("exports a lean FAST_TRIAGE_SYSTEM_PROMPT", () => {
|
||||
expect(typeof FAST_TRIAGE_SYSTEM_PROMPT).toBe("string");
|
||||
|
||||
@@ -144,7 +144,7 @@ Follow this structure exactly:
|
||||
|
||||
- [ ] {Specific, verifiable outcome}
|
||||
- [ ] {Specific, verifiable outcome}
|
||||
- [ ] Run targeted tests for changed files
|
||||
- [ ] Run targeted tests for changed files, asserting the invariant across all known surfaces (enumerate every provider/bridge, desktop + mobile breakpoints, and empty/undefined/populated data states)
|
||||
|
||||
**Artifacts:**
|
||||
- \`path/to/file\` (new | modified)
|
||||
@@ -220,6 +220,7 @@ files with assertions that run via a test runner. Typechecks and builds are NOT
|
||||
tests. Manual verification is NOT a test.
|
||||
|
||||
- Each implementation step should include writing tests for the code being changed
|
||||
- For bug fixes, regression tests must assert the invariant across all known surfaces — enumerate every provider/bridge, desktop + mobile breakpoints, and empty/undefined/populated data states — not just the reported repro (see FN-5787/FN-5789/FN-5803 and FN-5751)
|
||||
- The final Testing step runs lint, the FULL test suite, and project typecheck when the repo exposes one
|
||||
- Specs must instruct executors to fix lint failures and quality-gate failures directly, even when the required edits extend beyond the original File Scope
|
||||
- If the project has no test framework, the Testing step must include setting one up
|
||||
@@ -429,7 +430,7 @@ Follow this structure exactly:
|
||||
|
||||
- [ ] {Specific, verifiable outcome}
|
||||
- [ ] {Specific, verifiable outcome}
|
||||
- [ ] Run targeted tests for changed files
|
||||
- [ ] Run targeted tests for changed files, asserting the invariant across all known surfaces (enumerate every provider/bridge, desktop + mobile breakpoints, and empty/undefined/populated data states)
|
||||
|
||||
**Artifacts:**
|
||||
- \`path/to/file\` (new | modified)
|
||||
@@ -500,6 +501,7 @@ If this task REMOVES existing functionality (deleting modules, settings, API end
|
||||
## Testing requirements
|
||||
- Require real automated tests with assertions that run in the project's test runner
|
||||
- Typecheck/build/manual checks are not tests and cannot replace tests
|
||||
- For bug fixes, regression tests must assert the invariant across all known surfaces — enumerate every provider/bridge, desktop + mobile breakpoints, and empty/undefined/populated data states — not just the reported repro (see FN-5787/FN-5789/FN-5803 and FN-5751)
|
||||
- Include targeted tests in implementation steps and full quality-gate runs in final verification
|
||||
|
||||
## Duplicate check
|
||||
|
||||
Reference in New Issue
Block a user