feat(test): add flaky-test quarantine ledger with --diff ratchet exemption note
This commit is contained in:
@@ -16,6 +16,14 @@
|
||||
* remove (old path) + add (new path); the diff lists the removed ids so
|
||||
* the rename is reviewable. New ids in <after> never fail the diff.
|
||||
*
|
||||
* DELIBERATELY UNWIRED IN CI: the quarantine deletion ratchet
|
||||
* (scripts/lib/test-quarantine.json, docs/testing.md) deletes expired
|
||||
* quarantined tests by design, and a snapshot-based --diff guard would
|
||||
* fail on exactly those deletions. If --diff is ever wired to a
|
||||
* committed snapshot, it must exempt ledger-driven deletions (diff
|
||||
* against "snapshot minus quarantined entries"), or the two mechanisms
|
||||
* deadlock.
|
||||
*
|
||||
* --dashboard-curated
|
||||
* Assert that every `*.test.{ts,tsx}` file under packages/dashboard/app
|
||||
* and packages/dashboard/src is included by at least one *executed*
|
||||
|
||||
4
scripts/lib/test-quarantine.json
Normal file
4
scripts/lib/test-quarantine.json
Normal file
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"$comment": "Flaky-test quarantine ledger (deletion ratchet — see AGENTS.md 'Flaky tests: quarantine on sight' and docs/testing.md 'Quarantine ledger and the deletion ratchet'). A test observed failing without a corresponding real bug is quarantined ON SIGHT: add an entry here AND a matching one-line `exclude` entry in that package's vitest config, in the same commit. Every entry needs a non-empty `reason` (link the failing run) and a `quarantinedAt` date — the entry expires 14 days later, at which point the test file is DELETED unless someone rescues it with evidence it catches real regressions plus a root-cause fix (never appeasement). There is deliberately no loader module and no automation around this file: it is a dated record, the vitest config exclude is the mechanism, and the sweep is policy executed by whoever touches the suite.",
|
||||
"entries": []
|
||||
}
|
||||
Reference in New Issue
Block a user