Files
fusion/docs/solutions
gsxdsm 5792452f0a docs(workflow-learnings): mutation testing has one blind spot — your own imagination (#2858)
The most transferable thing this lane produced, and it is a correction
to advice I wrote earlier in the same document.

## The gap

Every other section here says *"watch the guard go red before you trust
it."* That rule is necessary and **not sufficient**, and the way it
fails cost the most.

Three instruments were written during this program. Each was
mutation-tested in both directions before shipping. Each was green.
Reviewers then found, in those same instruments:

- a **file-level pre-filter** that skipped whole files, so a forbidden
site added to a file with no other SQL was invisible;
- an **anchored pattern** that missed qualified and compound fragments
(`t."column" = 'done'`);
- a scan over **SOURCE text**, where a double-quoted TS string still
spells `\"column\"` with the backslashes in it;
- an operator list of `= != <>` that never considered **`IN (...)`**;
- and worst, a template scan that joined only the **static spans** — so
a Drizzle query, which puts the COLUMN in the interpolation hole and the
legacy id in the static text, matched nothing. **That gate was blind on
the exact files it was built to freeze.** Enabling that one shape took
the population from 14 to 31 and revealed five previously invisible
files.

## Why the mutation tests could not catch any of them

All five are **false negatives**, and the reason is structural rather
than sloppy:

> A mutation you write is a mutation you already imagined, so it lands
inside the space your scanner understands.

Reintroducing a defect the checker was designed around proves the
checker still handles that defect. It says nothing about shapes you
never modelled.

## What does find them

1. **Run the instrument against the code it was written for and read the
hits by hand.** The Drizzle blindness was obvious the moment someone
asked *"why is the merge-queue query — the reason this exists — not in
the output?"*
2. **Prefer one unanchored pattern over a fast pre-filter plus a precise
one.** Every false negative above came from two patterns disagreeing
about whether to run the real check at all. A pre-filter is a second,
weaker specification of the thing you are testing.
3. **Treat a guard's own count as a claim to verify, not a result.** "14
sites" read as coverage for days; it was the subset one scanner happened
to model.

A false positive is loud and gets fixed. **A false negative prints a
baseline and reads as coverage.**

Docs only — no changeset.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:55:00 -07:00
..