Files
fusion/docs/solutions/workflow-learnings
gsxdsm d14294b6cb docs(solutions): add the count-based probe, which the report-only trap cannot fool (#3257)
## What

Adds one technique to #3255. Docs only.

#3255 records that probing a ratchet **by exit code** can read green
because the tool is report-only without `--strict` — a real trap that
nearly got a healthy gate reported as dead. There is a second technique
that sidesteps it entirely and is strictly more informative: **parse the
tool's own per-file count.**

```bash
node scripts/check-move-target-literals.mjs 2>&1 | grep -a "my-probe-tmp" \
  | grep -aoE "^ +[0-9]+" | tr -d ' '
```

**Immune to the report-only trap** — a report-only run still *prints*
the count, so the number moves 0 → 1 whether or not `--strict` was
passed.

**It measures which shapes, not just whether something fired.** An exit
code is one bit for the whole run. Auditing a detector means asking *"of
these five spellings, which are seen?"*, and five separate binary runs
cannot distinguish **partial** detection from a probe file that failed
to compile. The move-target audit read `direct 1 / backtick 1 / ternary
0 / const 0` in a single run, which named the gap immediately.

## Both belong

| question | technique |
|---|---|
| **can this ratchet fail at all?** | `pnpm check:*` — ask this first
(#3255 §1) |
| **what can it see?** | per-file counts — an exit code is too coarse |

I also added a caveat that applies to both: confirm the probe is
actually being scanned by watching the tool's **scanned-file total**
move. A probe that never compiled and a probe the tool never discovered
both report zero hits, and neither is a finding — that one cost me a
wasted measurement before I noticed the total had stayed at 1961.

## Why this is worth a follow-up rather than a comment

#3255's rule as written — *"use `pnpm check:*`, not a bare `node
scripts/...`"* — would have made the shape-coverage audits impossible,
since `--strict` collapses five distinct per-form answers into one bit.
The rule is right for its question and wrong for the other one, and the
distinction is easy to lose once only the rule survives in someone's
memory.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
  * Added guidance for evaluating ratchets using per-file output counts.
* Documented report-only and shape-coverage limitations, count-based
versus failure-based checks, and verifying that probe files were
scanned.
  * Included a command example for probing ratchet behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 16:03:58 -07:00
..