Tests must assert behavior, not source text. A test that pins an FNXC block, a
date stamp, or comment prose guards documentation — and AGENTS.md tells authors
to keep those comments current, so the two rules fight and the test loses in the
worst way.
Measured today: grok-runtime-bootstrap.test.ts asserted runTaskMerge's body
contained "FNXC:GrokCliRouting 2026-07-15-10:17". FN-9167 legitimately rewrote
that function and dropped the block while leaving behavior intact; the test went
red, and the fix applied earlier in this sweep was to RE-ADD THE COMMENT to
packages/cli/src/commands/task.ts. A comment returned to shipped source not
because it documented anything true, but to appease a test. Four more such
assertions sat in dashboard CSS tests, each beside a real assertion, each adding
nothing.
- Drops the two prose pins from grok-runtime-bootstrap; its real structural
guard (`not.toContain("mergePluginRunner")`) stays. The product comment stays
too — it is accurate documentation, it was simply never a test's business.
- Adds scripts/check-no-comment-assertions-in-tests.mjs, wired into pretest,
pretest:full, and test:gate:static. It flags the unambiguous case; an earlier
draft that also matched `/*` produced 24 false positives and zero true ones,
because a regex cannot separate comment prose from a path glob.
- Adds the standing rule to AGENTS.md, with an explicit boundary: prose,
comments, and date stamps are never a test subject, while code-construct and
call-site-allowlist guards (no-blocking-shellout, vi-mock resolution, durable
write and emit-surface inventories, legacy tombstones) are a different
category and stay.
Also carries a product fix that the agent-generation tests surfaced: the
system prompt exists in two copies, and `resolvePrompt` returns core's catalog
default, so FN-021 adding the xhigh/max thinking levels to the dashboard copy
alone left every real generation run advertising levels that stop at "high".
Core's copy is synced and both literals now say they must move together.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
12c292ea6b