feat(test): add flaky-test quarantine ledger with --diff ratchet exemption note

This commit is contained in:
gsxdsm
2026-06-05 09:03:22 -07:00
parent fd7fed22f4
commit bbeb86e8f9
2 changed files with 12 additions and 0 deletions

View File

@@ -16,6 +16,14 @@
* remove (old path) + add (new path); the diff lists the removed ids so
* the rename is reviewable. New ids in <after> never fail the diff.
*
* DELIBERATELY UNWIRED IN CI: the quarantine deletion ratchet
* (scripts/lib/test-quarantine.json, docs/testing.md) deletes expired
* quarantined tests by design, and a snapshot-based --diff guard would
* fail on exactly those deletions. If --diff is ever wired to a
* committed snapshot, it must exempt ledger-driven deletions (diff
* against "snapshot minus quarantined entries"), or the two mechanisms
* deadlock.
*
* --dashboard-curated
* Assert that every `*.test.{ts,tsx}` file under packages/dashboard/app
* and packages/dashboard/src is included by at least one *executed*

View File

@@ -0,0 +1,4 @@
{
"$comment": "Flaky-test quarantine ledger (deletion ratchet — see AGENTS.md 'Flaky tests: quarantine on sight' and docs/testing.md 'Quarantine ledger and the deletion ratchet'). A test observed failing without a corresponding real bug is quarantined ON SIGHT: add an entry here AND a matching one-line `exclude` entry in that package's vitest config, in the same commit. Every entry needs a non-empty `reason` (link the failing run) and a `quarantinedAt` date — the entry expires 14 days later, at which point the test file is DELETED unless someone rescues it with evidence it catches real regressions plus a root-cause fix (never appeasement). There is deliberately no loader module and no automation around this file: it is a dated record, the vitest config exclude is the mechanism, and the sweep is policy executed by whoever touches the suite.",
"entries": []
}