Extends the doc merged in #3255 with two more instances of the same
pattern, both found this session, **neither involving a ratchet**. Four
instances now, from four unrelated directions:
| what was read as "pass" | what the green actually meant |
| --- | --- |
| `node scripts/check-*.mjs` exits 0 | report-only mode — the failure
path needs `--strict` |
| a census reports 0 for a new file | the file is untracked, so it was
never scanned |
| a backgrounded `cmd > log; grep …` reports exit 0 | that is `grep`'s
status; the suite inside had 8 failures |
| a rebased branch's tests pass | the rebase never started, so it ran on
the **old** base |
The two new ones are worth writing down because they are not about
tooling anyone built here — they are about how results are read.
**Exit codes belong to the last command in the pipeline.** A
backgrounded `run_tests > log 2>&1; echo done; grep X log` exits with
`grep`'s status, so the harness reported "completed, exit code 0" for a
dashboard suite that had 8 failures. I nearly recorded that suite as
green. Read the summary out of the log; never infer a suite's result
from a wrapper's exit code.
**A failed rebase leaves you on the old base, and the tests still pass
there.** `git rebase` refused with `cannot rebase: You have unstaged
changes`, so the branch never moved. `git diff origin/main` then listed
20+ files including other workers' commits — which reads exactly like my
branch had reverted their work — and a full test run on that tree came
back green. Both signals were true about a tree nobody cared about.
```
git merge-base --is-ancestor origin/main HEAD
```
said STALE while the tests said pass. That is the only check that
separates the two, and it belongs before any claim of "verified on
current main".
The shared tell, stated once: **a result too clean, or too alarming, for
what changed.** Every probe shape passing including ones that obviously
should not; a two-file branch appearing to revert twenty. When the
answer does not fit the size of the question, find out what was actually
measured before believing it.
## Verification
Docs only; no code paths change. `lifecycle-columns`,
`move-target-literals`, `inert-sync-lanes`, `quarantine-ledger` all exit
0. No changeset — AGENTS.md excludes internal docs.
**Pre-existing red, not from this branch:** `check:fnxc-future-dates`
currently fails on main from a `2026-08-01-00:50` stamp in
`packages/core/src/task-store/lifecycle-ops.ts` (commit `e52da740a5`) —
a timezone-ahead clock writing tomorrow's date, at 23:45 UTC. Already
claimed by **#3269 and #3270**, so I have not touched it; flagging only
so this branch's CI result is not misattributed. It is the same
recurring class this doc's sibling rule addresses: take the stamp from
`date -u`, not the local clock.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added guidance for identifying misleadingly successful CI and test
results.
* Documented checks for report-only runs, untracked files, masked
failures, and tests running on an outdated code base.
* Included recommendations for reviewing logs and verifying branch
ancestry.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
29eb512d57