docs(solutions): the general shape — a green that answers a different question (#3273)

Extends the doc merged in #3255 with two more instances of the same
pattern, both found this session, **neither involving a ratchet**. Four
instances now, from four unrelated directions:

| what was read as "pass" | what the green actually meant |
| --- | --- |
| `node scripts/check-*.mjs` exits 0 | report-only mode — the failure
path needs `--strict` |
| a census reports 0 for a new file | the file is untracked, so it was
never scanned |
| a backgrounded `cmd > log; grep …` reports exit 0 | that is `grep`'s
status; the suite inside had 8 failures |
| a rebased branch's tests pass | the rebase never started, so it ran on
the **old** base |

The two new ones are worth writing down because they are not about
tooling anyone built here — they are about how results are read.

**Exit codes belong to the last command in the pipeline.** A
backgrounded `run_tests > log 2>&1; echo done; grep X log` exits with
`grep`'s status, so the harness reported "completed, exit code 0" for a
dashboard suite that had 8 failures. I nearly recorded that suite as
green. Read the summary out of the log; never infer a suite's result
from a wrapper's exit code.

**A failed rebase leaves you on the old base, and the tests still pass
there.** `git rebase` refused with `cannot rebase: You have unstaged
changes`, so the branch never moved. `git diff origin/main` then listed
20+ files including other workers' commits — which reads exactly like my
branch had reverted their work — and a full test run on that tree came
back green. Both signals were true about a tree nobody cared about.

```
git merge-base --is-ancestor origin/main HEAD
```

said STALE while the tests said pass. That is the only check that
separates the two, and it belongs before any claim of "verified on
current main".

The shared tell, stated once: **a result too clean, or too alarming, for
what changed.** Every probe shape passing including ones that obviously
should not; a two-file branch appearing to revert twenty. When the
answer does not fit the size of the question, find out what was actually
measured before believing it.

## Verification

Docs only; no code paths change. `lifecycle-columns`,
`move-target-literals`, `inert-sync-lanes`, `quarantine-ledger` all exit
0. No changeset — AGENTS.md excludes internal docs.

**Pre-existing red, not from this branch:** `check:fnxc-future-dates`
currently fails on main from a `2026-08-01-00:50` stamp in
`packages/core/src/task-store/lifecycle-ops.ts` (commit `e52da740a5`) —
a timezone-ahead clock writing tomorrow's date, at 23:45 UTC. Already
claimed by **#3269 and #3270**, so I have not touched it; flagging only
so this branch's CI result is not misattributed. It is the same
recurring class this doc's sibling rule addresses: take the stamp from
`date -u`, not the local clock.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Added guidance for identifying misleadingly successful CI and test
results.
* Documented checks for report-only runs, untracked files, masked
failures, and tests running on an outdated code base.
* Included recommendations for reviewing logs and verifying branch
ancestry.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

docs/solutions/workflow-learnings/probe-the-instrument-the-way-ci-runs-it.md

@@ -127,6 +127,44 @@ both report zero hits, and neither is a finding.
   result: a path can appear under both flags in some index states). Ignored paths stay excluded, so
   build output does not leak into the count.
 
+## The general shape: a green that answers a different question
+
+The two defects above are instances of one thing, and it kept happening — **four times in a single
+session**, from four unrelated directions. Each time the signal was real; it was simply the answer to
+a question other than the one being asked.
+
+| what was read as "pass" | what the green actually meant |
+| --- | --- |
+| `node scripts/check-*.mjs` exits 0 | report-only mode — the failure path needs `--strict` |
+| a census reports 0 for a new file | the file is untracked, so it was never scanned |
+| a backgrounded `cmd > log; grep …` reports exit 0 | that is `grep`'s status; the suite inside had 8 failures |
+| a rebased branch's tests pass | the rebase never started, so it ran on the **old** base |
+
+The last two are worth spelling out because neither involves a ratchet at all.
+
+**Exit codes belong to the last command in the pipeline.** `run_tests > log 2>&1; echo done; grep X log`
+exits with `grep`'s status. A harness that reports "completed, exit 0" is reporting on the pipeline,
+not on the tests. Read the summary line out of the log; never infer a suite's result from a wrapper's
+exit code.
+
+**A failed rebase leaves you on the old base, and the tests still pass there.** `git rebase` refused
+with `cannot rebase: You have unstaged changes` — so the branch never moved. `git diff origin/main`
+then listed 20+ files including other workers' commits, which reads exactly like the branch had
+reverted their work, and a full test run on that tree came back green. Both signals were true about a
+tree nobody cared about.
+
+```sh
+git merge-base --is-ancestor origin/main HEAD    # the only check that distinguishes the two
+```
+
+It said STALE while the tests said pass. Run it after any rebase, before trusting a verification —
+especially before reporting "verified on current main".
+
+**The tell, in all four cases, is a result that is too clean or too alarming for what changed.** Every
+probe shape passing, including ones that obviously should not. A two-file branch that appears to
+revert twenty. When the answer does not fit the size of the question, check what was actually
+measured before believing it.
+
 ## Provenance
 
 Both defects were found by probing, not by reading: staged probe files measured against the tools with