root/fusion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test: fix the dashboard quality runner's self-spawning tests

Browse Source 
Running the dashboard's REAL test command (`run-quality-tests.mjs`, which shards
into 15 lanes) surfaced two failures that a plain `vitest run` never shows —
worth noting on its own, since measuring around a package's own command is how
a suite gets called green on a number the project does not produce.

`scripts/__tests__/run-quality-tests.test.ts` spawns the package's own
`pnpm --filter @fusion/dashboard test` to prove the package-command wiring. Inside
a full lane run that child inherited a Corepack environment that stopped to ask
about downloading pnpm, so it never launched, the lane log came back empty, and
the assertion read 0 launched projects. It passed in isolation only because that
shell had already resolved pnpm. Both spawn sites now set
COREPACK_ENABLE_DOWNLOAD_PROMPT=0; nothing about the assertions changed.

Also records PlanningModeModal.planning-flow as a suite-only flake rather than
forcing it green: it fails only in lane `app:backfill-3` under four concurrent
6GB shards, passes 83/83 in isolation, and picked a DIFFERENT case on each of two
runs — a render-settle timing problem, not a product defect. Recorded as a first
sighting; a repeat of the same case is an on-sight quarantine.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
gsxdsm
2026-08-23 18:03:38 -07:00
parent 097fa9b403
commit 5378bca7f6
2 changed files with 52 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
docs/solutions/test-failures/suite-only-flakes-observed-register.md
View File

@@ -524,3 +524,35 @@ reconciler tests that fail only alongside other suites, which points at shared f
cross-file state rather than a product defect. No timeout was widened, no retry added, and no
assertion relaxed. A SECOND sighting is an ordinary on-sight quarantine with no further discretion,
per the standing rule in AGENTS.md.
---
## Entry: `PlanningModeModal.planning-flow` under dashboard lane sharding (first sighting)
- **File:** `packages/dashboard/app/components/__tests__/PlanningModeModal.planning-flow.test.tsx`
- **Exact tests:** a DIFFERENT case failed on each of two consecutive full-lane runs —
`PlanningModeModal sequential flow > keeps the newer session when delayed duplicate reconciliation returns 'a durable question' on 'mobile'`, then
`PlanningModeModal sequential flow > can refine a stopped initial plan into the first question`.
- **Owner:** unowned — first sighting for this file. Recorded rather than quarantined: the file carries 83 tests and quarantine is file-level.
- **Observed tree/SHA:** `c82e420ba0`, via the package's real command `pnpm --filter @fusion/dashboard test` (the `run-quality-tests.mjs` lane runner), lane `app:backfill-3` (`--project dashboard-app-quality-backfill --shard=3/4`), concurrency 2, 6144MiB heap per lane.
- **Observed frequency:** twice in two full-lane runs, each time a different case; passes 83/83 in isolation every time.
Verbatim observed failure (second run):
```
FAIL |dashboard-app-quality-backfill| app/components/__tests__/PlanningModeModal.planning-flow.test.tsx > PlanningModeModal sequential flow > can refine a stopped initial plan into the first question
TestingLibraryElementError: Unable to find an element by: [data-testid="planning-plan-review"]
```
| run | result |
|---|---|
| lane runner, default (fail-fast), `c82e420ba0` | **failed** on the delayed-duplicate-reconciliation case |
| lane runner, `--all --no-fail-fast`, same tree | **failed** on the refine-stopped-plan case |
| isolated `vitest run <file>`, same tree, repeatedly | **passed** 83/83 |
The moving target plus a "cannot find element" shape points at render/settle timing under a loaded
shard, not a product defect — a wait that is adequate on an idle machine and not under four
concurrent 6GB lanes. No timeout was widened, no retry added, no assertion relaxed. A SECOND sighting
of the *same* case is an ordinary on-sight quarantine per the standing rule in AGENTS.md; because the
case moves, the honest rescue is a deterministic settle signal in this file's harness rather than a
longer wait.

20
packages/dashboard/scripts/__tests__/run-quality-tests.test.ts
View File

@@ -167,6 +167,16 @@ describe("dashboard quality orchestrator", () => {
env: {
...process.env,
npm_config_ignore_scripts: "true",
/*
FNXC:QualityRunnerSelfTest 2026-08-23-19:20:
These cases spawn the package's OWN `pnpm ... test` command, so the child inherits Corepack.
Inside a full lane run the child hit Corepack's interactive prompt
("! Corepack is about to download .../pnpm-10.33.0.tgz"), never launched, and the lane log
came back empty — the assertion then read 0 launched projects and failed. It passed when the
file was run alone only because that shell had already resolved pnpm. Disable the prompt so
the child resolves pnpm non-interactively; this changes nothing about what is asserted.
*/
COREPACK_ENABLE_DOWNLOAD_PROMPT: "0",
FUSION_DASHBOARD_TEST_CONCURRENCY: "1",
FUSION_DASHBOARD_QUALITY_TEST_MODE: "1",
FUSION_DASHBOARD_QUALITY_RUNNER: fixtureRunnerPath,
@@ -202,6 +212,16 @@ describe("dashboard quality orchestrator", () => {
env: {
...process.env,
npm_config_ignore_scripts: "true",
/*
FNXC:QualityRunnerSelfTest 2026-08-23-19:20:
These cases spawn the package's OWN `pnpm ... test` command, so the child inherits Corepack.
Inside a full lane run the child hit Corepack's interactive prompt
("! Corepack is about to download .../pnpm-10.33.0.tgz"), never launched, and the lane log
came back empty — the assertion then read 0 launched projects and failed. It passed when the
file was run alone only because that shell had already resolved pnpm. Disable the prompt so
the child resolves pnpm non-interactively; this changes nothing about what is asserted.
*/
COREPACK_ENABLE_DOWNLOAD_PROMPT: "0",
FUSION_DASHBOARD_TEST_CONCURRENCY: "1",
FUSION_DASHBOARD_QUALITY_TEST_MODE: "1",
FUSION_DASHBOARD_QUALITY_RUNNER: fixtureRunnerPath,