Files
fusion/docs/solutions
gsxdsm 478b15d7ec docs(solutions): add the CI failure-rate method, and a fourth instance (#3244)
## What

Follow-up to #3243. That note said *"take a second measurement of a
different kind"* — true, and useless at 2am without the technique. This
adds the one that actually settled every case, plus a fourth instance
that occurred after #3243 was written. Docs only.

## The technique

Enumerate recent failing CI runs and compute a **per-file failure
rate**. Seven runs separated three populations that are
indistinguishable from a single local run:

| rate on CI | meaning | action |
|---|---|---|
| **7/7** | consistent, real | fix, or diagnose and hand off with
evidence |
| **1/7** | intermittent | flake or race; two in one subsystem is a
product-race smell |
| **0/7** (fails only locally) | environment | fix your sandbox, change
**nothing** in the repo |

Measured on this repo's main while writing it: `planning-browser-e2e`
**7/7**, `postgres/schema-applier` **1/7**, `report-store.pg` **1/7**.

## The fourth instance

#3243 documented three reversals. A fourth happened after it merged: a
component test with **2 failing cases locally, 0/7 on CI**. That makes
**three separate local-only failures in a single session** — a
model-routes test hanging offline, a component test with four failing
cases, and a set of assertions I was ready to call a regression.

Each felt like a finding. All three were my sandbox. That is frequent
enough to be a habit rather than bad luck, which is why it is worth a
row in a table rather than a mention.

## The cost asymmetry, which should drive the default

Acting on a **0/7** by quarantining **deletes coverage that is green
everywhere else**. Acting on a **7/7** by investigating costs an hour.
The errors are not symmetric, so when unsure which row you are in, the
cheap move is always more samples from the *other* environment — not
more confidence about the one you have.

This is the concrete form of the point the standing quarantine rule
already encodes with *"without a corresponding real bug"*: **"I saw it
fail" is not that clause**, and the failure-rate table is how you tell
the difference before acting.

```
lint clean; fnxc-future-dates: none added (exit code checked before piping)
```
2026-07-31 14:12:51 -07:00
..