From 5378bca7f61c83ab6bc6bf10858ef1e8c4e7215e Mon Sep 17 00:00:00 2001 From: gsxdsm Date: Sun, 23 Aug 2026 18:03:38 -0700 Subject: [PATCH] test: fix the dashboard quality runner's self-spawning tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Running the dashboard's REAL test command (`run-quality-tests.mjs`, which shards into 15 lanes) surfaced two failures that a plain `vitest run` never shows — worth noting on its own, since measuring around a package's own command is how a suite gets called green on a number the project does not produce. `scripts/__tests__/run-quality-tests.test.ts` spawns the package's own `pnpm --filter @fusion/dashboard test` to prove the package-command wiring. Inside a full lane run that child inherited a Corepack environment that stopped to ask about downloading pnpm, so it never launched, the lane log came back empty, and the assertion read 0 launched projects. It passed in isolation only because that shell had already resolved pnpm. Both spawn sites now set COREPACK_ENABLE_DOWNLOAD_PROMPT=0; nothing about the assertions changed. Also records PlanningModeModal.planning-flow as a suite-only flake rather than forcing it green: it fails only in lane `app:backfill-3` under four concurrent 6GB shards, passes 83/83 in isolation, and picked a DIFFERENT case on each of two runs — a render-settle timing problem, not a product defect. Recorded as a first sighting; a repeat of the same case is an on-sight quarantine. Co-Authored-By: Claude Opus 5 --- .../suite-only-flakes-observed-register.md | 32 +++++++++++++++++++ .../__tests__/run-quality-tests.test.ts | 20 ++++++++++++ 2 files changed, 52 insertions(+) diff --git a/docs/solutions/test-failures/suite-only-flakes-observed-register.md b/docs/solutions/test-failures/suite-only-flakes-observed-register.md index 4fe5f84264..c55ace6111 100644 --- a/docs/solutions/test-failures/suite-only-flakes-observed-register.md +++ b/docs/solutions/test-failures/suite-only-flakes-observed-register.md @@ -524,3 +524,35 @@ reconciler tests that fail only alongside other suites, which points at shared f cross-file state rather than a product defect. No timeout was widened, no retry added, and no assertion relaxed. A SECOND sighting is an ordinary on-sight quarantine with no further discretion, per the standing rule in AGENTS.md. + +--- + +## Entry: `PlanningModeModal.planning-flow` under dashboard lane sharding (first sighting) + +- **File:** `packages/dashboard/app/components/__tests__/PlanningModeModal.planning-flow.test.tsx` +- **Exact tests:** a DIFFERENT case failed on each of two consecutive full-lane runs — + `PlanningModeModal sequential flow > keeps the newer session when delayed duplicate reconciliation returns 'a durable question' on 'mobile'`, then + `PlanningModeModal sequential flow > can refine a stopped initial plan into the first question`. +- **Owner:** unowned — first sighting for this file. Recorded rather than quarantined: the file carries 83 tests and quarantine is file-level. +- **Observed tree/SHA:** `c82e420ba0`, via the package's real command `pnpm --filter @fusion/dashboard test` (the `run-quality-tests.mjs` lane runner), lane `app:backfill-3` (`--project dashboard-app-quality-backfill --shard=3/4`), concurrency 2, 6144MiB heap per lane. +- **Observed frequency:** twice in two full-lane runs, each time a different case; passes 83/83 in isolation every time. + +Verbatim observed failure (second run): + +``` +FAIL |dashboard-app-quality-backfill| app/components/__tests__/PlanningModeModal.planning-flow.test.tsx > PlanningModeModal sequential flow > can refine a stopped initial plan into the first question +TestingLibraryElementError: Unable to find an element by: [data-testid="planning-plan-review"] +``` + +| run | result | +|---|---| +| lane runner, default (fail-fast), `c82e420ba0` | **failed** on the delayed-duplicate-reconciliation case | +| lane runner, `--all --no-fail-fast`, same tree | **failed** on the refine-stopped-plan case | +| isolated `vitest run `, same tree, repeatedly | **passed** 83/83 | + +The moving target plus a "cannot find element" shape points at render/settle timing under a loaded +shard, not a product defect — a wait that is adequate on an idle machine and not under four +concurrent 6GB lanes. No timeout was widened, no retry added, no assertion relaxed. A SECOND sighting +of the *same* case is an ordinary on-sight quarantine per the standing rule in AGENTS.md; because the +case moves, the honest rescue is a deterministic settle signal in this file's harness rather than a +longer wait. diff --git a/packages/dashboard/scripts/__tests__/run-quality-tests.test.ts b/packages/dashboard/scripts/__tests__/run-quality-tests.test.ts index b5573c3f32..4dc24583d1 100644 --- a/packages/dashboard/scripts/__tests__/run-quality-tests.test.ts +++ b/packages/dashboard/scripts/__tests__/run-quality-tests.test.ts @@ -167,6 +167,16 @@ describe("dashboard quality orchestrator", () => { env: { ...process.env, npm_config_ignore_scripts: "true", + /* + FNXC:QualityRunnerSelfTest 2026-08-23-19:20: + These cases spawn the package's OWN `pnpm ... test` command, so the child inherits Corepack. + Inside a full lane run the child hit Corepack's interactive prompt + ("! Corepack is about to download .../pnpm-10.33.0.tgz"), never launched, and the lane log + came back empty — the assertion then read 0 launched projects and failed. It passed when the + file was run alone only because that shell had already resolved pnpm. Disable the prompt so + the child resolves pnpm non-interactively; this changes nothing about what is asserted. + */ + COREPACK_ENABLE_DOWNLOAD_PROMPT: "0", FUSION_DASHBOARD_TEST_CONCURRENCY: "1", FUSION_DASHBOARD_QUALITY_TEST_MODE: "1", FUSION_DASHBOARD_QUALITY_RUNNER: fixtureRunnerPath, @@ -202,6 +212,16 @@ describe("dashboard quality orchestrator", () => { env: { ...process.env, npm_config_ignore_scripts: "true", + /* + FNXC:QualityRunnerSelfTest 2026-08-23-19:20: + These cases spawn the package's OWN `pnpm ... test` command, so the child inherits Corepack. + Inside a full lane run the child hit Corepack's interactive prompt + ("! Corepack is about to download .../pnpm-10.33.0.tgz"), never launched, and the lane log + came back empty — the assertion then read 0 launched projects and failed. It passed when the + file was run alone only because that shell had already resolved pnpm. Disable the prompt so + the child resolves pnpm non-interactively; this changes nothing about what is asserted. + */ + COREPACK_ENABLE_DOWNLOAD_PROMPT: "0", FUSION_DASHBOARD_TEST_CONCURRENCY: "1", FUSION_DASHBOARD_QUALITY_TEST_MODE: "1", FUSION_DASHBOARD_QUALITY_RUNNER: fixtureRunnerPath,