Commit Graph

32 Commits

Author SHA1 Message Date
gsxdsm
f42b49ab2d FN-7105: add CI print-mode regression guard
Add regression coverage ensuring CI=true no longer forces test-changed into full-suite mode.

- Add a reusable --print-mode subprocess helper for synthetic test-changed repositories.
- Seed the synthetic repo with changeset config so mode decisions match workspace expectations.
- Assert CI=true stays in gate or changed mode while --full and FUSION_TEST_FULL still select full mode.

Files changed:
 scripts/__tests__/test-changed.test.mjs | 54 +++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

Fusion-Task-Id: FN-7105

Fusion-Task-Lineage: f4a26d96-3992-4f25-a456-04f2b9b8a4dc

Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
2026-06-27 09:28:30 -07:00
gsxdsm
1699c60033 fix: anchor rootDir at the git toplevel so subdir runs don't skip tests (#1785)
greptile: my earlier existence-check anchor was necessary but insufficient.
`rootDir` (= process.cwd() when FUSION_PROJECT_DIR is unset) drives ALL workspace
discovery (readWorkspacePatterns / listWorkspacePackageInfos /
packageHasVitestConfig). Launched from a package subdir, cwd-based discovery
found no packages, so decideExecutionPlan saw "no affected package", ran only the
gate, and exited successfully WITHOUT running the live changed package tests.

Fix the root cause: resolveRepoRoot() resolves the git toplevel as the fallback
(FUSION_PROJECT_DIR still the explicit override; cwd only when git can't report a
toplevel). This is correct from any cwd inside the repo, including a git worktree
(how the engine runs per-task verification). repoRootForExistence is now
redundant and removed; the existence check defaults back to rootDir.

Demonstrated: resolveRepoRoot() from packages/core (no FUSION_PROJECT_DIR) now
resolves the repo root and finds the workspace. +1 regression test. 121/121.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 22:56:10 -07:00
gsxdsm
e29954e50d Merge remote-tracking branch 'origin/main' into feature/tasks-take-too-long
# Conflicts:
#	scripts/__tests__/test-changed.test.mjs
2026-06-25 22:41:18 -07:00
gsxdsm
c36f234da2 Address PR review feedback (#1785)
- P2 (greptile): anchor the changed-test existence check at the git repo root
  (repoRootForExistence via `git rev-parse --show-toplevel`) instead of rootDir,
  so a script run from a package subdir without FUSION_PROJECT_DIR no longer
  forms a doubled path and silently drops live tests into the delegate path.
  +1 regression test (default root resolves to repo root).
- coderabbit: fix stale "1-worker lane" wording in the delegation log (now
  "heavy memory-envelope lane") and the "single-worker envelope" test title,
  both stale after the 1->4 worker change.

test-changed 118/118, eslint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 22:30:57 -07:00
gsxdsm
2cff1864c9 fix: bound @fusion/core affected lane + tighten changed watchdog under engine kill
Three changes to make `pnpm test` reliably minimal and fail gracefully:

- @fusion/core is now a memory-envelope/wide-fan-out package (was unguarded).
  It's the hub nearly everything imports (~354 test files), so a core source
  edit made `vitest --changed` expand to ~the whole core suite and blow past the
  engine's 15-min verification kill -> SIGKILL + task restart. Adding it to
  SCOPED_AFFECTED_MEMORY_ENVELOPES applies the wide-fan-out guard (run only
  directly-changed core tests, else delegate) and the bounded env. core is NOT
  gate-covered, so delegation warns loudly rather than false-greens.

- Lower CLASS_BUDGET_BANDS.changed ceiling 20min -> 13min so the script watchdog
  fails a runaway local lane itself (exit 124, no restart) BEFORE the engine's
  15-min kill restarts the whole task. A tightening, not a timeout-widening.
  Guard test pins ceiling < 900_000ms.

- Raise scoped-affected worker fan-out 1 -> 4 (operator decision). Was 1 only
  for OOM safety (FN-6854/FN-6874); the fan-out guard now bounds the set so the
  hundreds-of-files OOM driver no longer reaches these workers. Heap stays
  6144MB/worker (~4x6GB on the lane) — revisit if a RAM-constrained CI runner
  OOMs. Trades FN-5048 worker-knob guidance for throughput, scoped to the
  bounded affected lanes only.

Tests: test-changed 117/117, watchdog 15/15, eslint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 21:42:22 -07:00
gsxdsm
8297762eeb Address PR review feedback (#1780)
- P2: filter directly-changed test files to paths that still exist on disk
  (existingChangedTestFilesInPackage) so deleted/renamed .test paths from
  `git diff` never reach `vitest run` positionally; all-deletions diff falls
  into the delegate-to-gate path.
- P1: make heavy-package delegation gate-coverage-aware
  (GATE_COVERED_MEMORY_ENVELOPE_PACKAGES). Engine delegation keeps the accurate
  "curated engine-core subset ran above" note; dashboard delegation now warns
  that the gate runs no dashboard tests and names the CI full-suite backstop,
  so the coverage gap is loud instead of a silent false-green.
- +4 regression tests (115/115).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 20:17:56 -07:00
gsxdsm
8754e28868 FN-7026: cap scoped affected test watchdogs
Keep changed-only affected test lanes from outliving the workspace verification timeout.

- Add a scoped affected watchdog ceiling below the default executor timeout.
- Use the ceiling for scoped dashboard and engine affected Vitest lanes while leaving full fallback budget unchanged.
- Cover the timeout derivation and dashboard-only package selection in script tests.
- Document the changed-only watchdog behavior for future test infrastructure changes.

Files changed:
 docs/testing.md                         |  6 +++--
 scripts/__tests__/test-changed.test.mjs | 48 +++++++++++++++++++++++++++++++++
 scripts/test-changed.mjs                | 18 ++++++++++---
 3 files changed, 66 insertions(+), 6 deletions(-)

Fusion-Task-Id: FN-7026

Fusion-Task-Lineage: 774519e0-505b-4e40-8309-5d2250da1eba
2026-06-25 19:35:46 -07:00
gsxdsm
c5fbe08c00 fix: bound changed-affected test lane to avoid vitest --changed graph blowup
`pnpm test`'s changed-affected lane ran `vitest run --changed <base>` for the
heavy packages (@fusion/engine, @fusion/dashboard). `vitest --changed` does
unbounded transitive module-graph expansion: one changed hub source file
selects ~8,393 test entries (79s just to list), which at the OOM-pinned
workers=1 exceeds the engine's 15-min VERIFICATION_TIMEOUT_WORKSPACE_MS. The
engine SIGKILLs and restarts the task, producing the observed loop of nine
15-min verification timeouts (~2.8h) on a single task.

Guard the lane with a git-only predictor: when a heavy package has changed
non-test source in its graph, run only the directly-changed test files;
when no test files changed, delegate cross-cutting coverage to the merge gate
(already run in changed mode). Test-only diffs keep normal --changed.
Mirrors the existing reverse-dependent blast cap one level down. No widened
timeouts, retries, or worker bumps. test:full remains the explicit full sweep.

Bounded engine run: 2.79s vs 79s. Regression suite 111/111.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 19:28:15 -07:00
gsxdsm
1e81f99f72 FN-6877: stabilize dashboard changed-mode affected runs
Stabilize changed-mode affected runs by giving dashboard its own memory envelope.

- Generalize scoped affected memory-envelope handling beyond the engine lane.
- Run @fusion/dashboard changed-mode Vitest scopes in a 6144 MB, single-worker envelope.
- Cover dashboard and engine scoped partitioning/env contracts with script tests.
- Document the dashboard changed-mode OOM/SIGKILL guard and remeasurement expectations.

Files changed:
 docs/testing.md                         |   6 +-
 scripts/__tests__/test-changed.test.mjs | 112 +++++++++++++++++++++++++++-----
 scripts/test-changed.mjs                |  67 +++++++++++++++----
 3 files changed, 155 insertions(+), 30 deletions(-)

Fusion-Task-Id: FN-6877

Fusion-Task-Lineage: 084df464-ebda-4578-bfac-305800da5b6c
2026-06-21 16:48:20 -07:00
gsxdsm
d57538581d FN-6854: isolate engine affected tests
Stabilize changed-test runs by giving the engine affected Vitest lane its own memory envelope.

- Split @fusion/engine out from other scoped affected-package Vitest runs.
- Apply a capped heap and single-worker settings to the isolated engine lane while preserving watchdog timing.
- Cover the partitioning and environment behavior with script tests and document the envelope.

Files changed:
 docs/testing.md                         |  3 +++
 scripts/__tests__/test-changed.test.mjs | 45 +++++++++++++++++++++++++++++++++
 scripts/test-changed.mjs                | 40 ++++++++++++++++++++++++++---
 3 files changed, 84 insertions(+), 4 deletions(-)

Fusion-Task-Id: FN-6854

Fusion-Task-Lineage: 144acf15-4ccb-4974-b06c-99cdcc2814dd
2026-06-21 13:25:41 -07:00
gsxdsm
ce90cc9b62 fix(FN-5048): keep Fusion test verification bounded 2026-06-21 10:05:12 -07:00
gsxdsm
24ff12471f Stop quarantine-list edits from forcing pnpm test into gate mode
scripts/lib/test-quarantine.json is runtime data (which tests are
quarantined), not executable test infra, but it tripped the shared-infra
catch-all in isSharedInfraChange. That forced mode=gate, which runs only
the fixed engine-core + cli-shape gate suite and returns before the
affected packages -- so a dev's real changes (e.g. @fusion/core,
@fusion/dashboard) got zero coverage whenever they also touched the
quarantine list. Classify the file as test-irrelevant so the diff stays
in changed mode and the affected packages run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:05:49 -07:00
gsxdsm
80fbcdd5a3 FN-6396: harden test worker temp cleanup
Prevent stale Fusion test worker roots from leaking across merge-gate runs.

- Add per-run tokens to worker-root owner markers and pruning checks.
- Remove self-minted fallback worker roots during Vitest exit cleanup.
- Cover stale pid reuse, markerless redir roots, and SIGKILL-style orphan pruning with regression tests.

Files changed:
 packages/core/src/__test-utils__/vitest-setup.ts   | 83 +++++++++++++++++++---
 .../core/src/__test-utils__/vitest-teardown.ts     |  8 ++-
 .../vitest-teardown-worker-root-cleanup.test.ts    | 19 ++++-
 scripts/__tests__/test-changed.test.mjs            | 74 ++++++++++++++++++-
 scripts/test-changed.mjs                           | 76 ++++++++++++++++----
 5 files changed, 233 insertions(+), 27 deletions(-)

Fusion-Task-Id: FN-6396

Fusion-Task-Lineage: 711d966d-c70e-4cd7-81cd-accd18f17202
2026-06-13 16:30:28 -07:00
gsxdsm
97a49ac196 FN-6382: unquarantine stabilized flaky tests
Restore quarantined tests by fixing their flaky harness seams instead of extending the deletion ratchet.

- Mark active Vitest worker roots and skip live worker roots during prune cleanup.
- Make bubblewrap backend coverage deterministic with an injectable runner and restore it to the engine gate.
- Remove rescued core and bubblewrap tests from the quarantine ledger and Vitest excludes.

Files changed:
 .../core/src/__test-utils__/vitest-teardown.ts     | 10 +++++-
 packages/core/vitest.config.ts                     |  8 +----
 .../__tests__/sandbox/bubblewrap-backend.test.ts   | 29 +++++++++------
 packages/engine/src/sandbox/bubblewrap-backend.ts  |  9 +++--
 packages/engine/vitest.config.ts                   |  1 -
 scripts/__tests__/test-changed.test.mjs            | 23 ++++++++++++
 scripts/lib/test-quarantine.json                   | 34 ++----------------
 scripts/test-changed.mjs                           | 41 ++++++++++++++++++++++
 8 files changed, 102 insertions(+), 53 deletions(-)

Fusion-Task-Id: FN-6382

Fusion-Task-Lineage: 018dc7ac-1ef1-495e-a5fd-96ea44fcd43b
2026-06-13 11:39:37 -07:00
gsxdsm
73be9f8c5d FN-6371: retry stale test worker pruning
Adds bounded retry handling so stale Fusion test temp roots are reclaimed instead of leaking after transient removal failures.

- Share prefix-scoped pruning between isolated HOME and worker temp roots.
- Retry rm-rf on transient busy/non-empty failures and treat ENOENT as successful cleanup.
- Warn once with bounded child diagnostics after persistent prune failures.
- Cover worker and home pruning retry, failure, and ENOENT behavior in script tests.

Files changed:
 scripts/__tests__/test-changed.test.mjs | 118 ++++++++++++++++++++++++++++++++
 scripts/test-changed.mjs                |  84 ++++++++++++++---------
 2 files changed, 168 insertions(+), 34 deletions(-)

Fusion-Task-Id: FN-6371

Fusion-Task-Lineage: a81e6849-5eb0-4bc0-8030-9f8eaed4e35a
2026-06-13 09:49:23 -07:00
gsxdsm
10972bbdce FN-6360: clean up leaked test worker temp roots
Ensure test isolation removes stale worker temp roots after interrupted or busy Vitest runs.

- Add bounded retry cleanup for Vitest worker roots during teardown.
- Prune orphaned fusion-test-workers-* directories before changed-test isolation checks.
- Cover worker-root retry and pruning behavior with targeted tests.

Files changed:
 .../core/src/__test-utils__/vitest-teardown.ts     | 50 ++++++++++--
 .../vitest-teardown-worker-root-cleanup.test.ts    | 88 ++++++++++++++++++++++
 scripts/__tests__/test-changed.test.mjs            | 33 ++++++++
 scripts/test-changed.mjs                           | 32 ++++++++
 4 files changed, 196 insertions(+), 7 deletions(-)

Fusion-Task-Id: FN-6360

Fusion-Task-Lineage: d537941d-dd58-403a-a82e-f0aeee9c1eb0
2026-06-13 08:11:46 -07:00
gsxdsm
5aa6cad168 Address PR review feedback (#1453)
- boot-smoke: shutdown verdict now requires SIGTERM actually delivered and a
  clean exit (code 0 or SIGTERM); EADDRINUSE port race retries with a fresh
  port (3 attempts)
- test-changed: rename shouldForceFullSuite -> isSharedInfraChange (it routes
  to gate mode, not full); run the changed-mode gate under the isolation guard
- workflows: least-privilege permissions (contents: read) on pr-checks and
  full-suite
2026-06-05 13:58:34 -07:00
gsxdsm
3387ac8096 fix(review): apply autofix feedback 2026-06-05 09:23:15 -07:00
gsxdsm
5a4bb9eeeb feat(test): pnpm test runs gate + affected set; remove implicit full-suite escalation (the local OOM path)
- decideExecutionPlan: implicit wide-blast reasons (missing base, diff failed,
  no changes, shared infra, unmapped package) route to new gate mode instead of full
- CI===true force-full branch removed (CI no longer calls test-changed.mjs)
- changed mode runs pnpm test:gate before the affected set
- full suite reachable only via explicit --full / FUSION_TEST_FULL=1
- characterization tests updated + new only-explicit-full invariant test
2026-06-05 09:08:20 -07:00
gsxdsm
2c41695e1f fix(test): dependency-aware and dirty-aware test cache invalidation
- cache key v2 = own hash + sorted transitive workspace dep hashes + shared inputs (lockfile, tsconfig.base, core __test-utils__ tree)
- working-tree-dirty files hashed by content (fixes false cache HIT on unstaged edits)
- core __test-utils__ folded globally: 16+ packages import it without a workspace dep on core
- dep folding adds ~5ms to the inner loop (memoized own-hashes)
- docs: cache semantics, --no-cache / FUSION_TEST_NO_CACHE, TTL rationale
2026-06-03 18:06:05 -07:00
gsxdsm
211c0fd557 perf(test): cut inner-loop fixed overhead to sub-second on cache-fresh runs
- skill-sync check conditioned on content hash of its inputs (skips ~0.3s spawn)
- ensure-test-artifacts: git-blob content-hash staleness; branch switches no longer trigger spurious ~2.6s tsc rebuilds (mtime fallback when dirty)
- isolation guard: cheap --before-fast reusing prior post-run baseline (~2.1s -> ~0.07s); detection proven preserved via injected-leak failure test
- vitest-setup: CI skips 4040-4045 discovery probe unless FUSION_RESERVED_PORTS set; kill-guard wrapper untouched, asymmetry pinned by port-probe-policy tests
- cache-fresh fast path skips sync/artifacts/HOME-prune entirely (mode line: fast-path=cache-fresh)
2026-06-03 17:57:29 -07:00
gsxdsm
c69d384e67 perf(test): add timing telemetry, cold-start probe, and baseline snapshot
- ci-test-shard.mjs: --write-timings aggregation into scripts/test-timings.json (bucketed, newer-snapshot-protected, corrupt-shard tolerant) and --cold-start-probe; CI shard invocations emit vitest json timings
- test-changed.mjs: structured mode/reason telemetry line (+ --print-mode)
- pr-checks.yml: upload per-shard timing artifacts
- baseline: docs/test-speed-baseline-2026-06-03.md (core 41s, engine 179s, cli 49s; cold-start ~1.3-1.8s/process => U8 gate: worthwhile-not-urgent)
2026-06-03 16:48:50 -07:00
Fusion (runfusion.ai)
82c8bced60 feat(FN-5144): add create-pr metadata and preflight routes
Added GitHub create-PR metadata and preflight routes to the dashboard, wiring up `submit-github.pr-options-preflight-metadata` contract and integration tests for the new preflight/confirm flow alongside supporting test configuration updates.

Fusion-Task-Id: FN-5144
2026-05-19 10:33:15 -07:00
Fusion (runfusion.ai)
15bb84f862 feat(FN-5124): add soft-delete QA boundary regression coverage
Adds a regression test for FN-5124 covering soft-delete QA boundary cases, targeting the engine reliability interaction suite. The single new test file exercises edge conditions in the soft-delete path to guard against future regressions.

Fusion-Task-Id: FN-5124

Fusion-Task-Lineage: 7166e252-9c98-4f51-956a-1402dae6f402
2026-05-19 05:52:09 -07:00
Fusion
2ba9b00704 feat(FN-3723): harden isolated HOME cleanup and shell connection visibility
- Harden test isolation cleanup retries and allow-list behavior in scripts/test-changed.mjs
- Expand test coverage for cleanup retry logic and allow-list handling in scripts/__tests__/test-changed.test.mjs
- Add shell-native connection status plumbing and UI coverage across dashboard, desktop preload/types, and mobile typings
- Add ShellConnectionStatus and mobile nav wiring plus related app/component test coverage
- Document shell connection contracts and add FN-3568 diagnosis/recovery notes

Fusion-Task-Id: FN-3723
2026-05-08 01:16:54 -07:00
Aron Prins
aaa4c59c54 test(FN-000): include reverse dependents in changed tests 2026-05-07 18:35:32 +02:00
Aron Prins
10ebfea17f test(FN-000): fix changed-test package cache dirs 2026-05-07 18:26:48 +02:00
Aron Prins
1ed390052c test(FN-000): address PR review feedback 2026-05-07 17:32:21 +02:00
Fusion
e7884370f2 fix(FN-3610): isolate test home for changed-package test runs
- Create a disposable HOME/USERPROFILE env for scripts/test-changed.mjs execution
- Run isolation guard checks with the same isolated env, including cache-hit no-op paths
- Clean up temp HOME after test execution to avoid residue
- Update test coverage and contributing docs for the shared isolation behavior

Fusion-Task-Id: FN-3610
2026-05-06 18:57:37 -07:00
Fusion
f6394cc22b test(FN-3607): harden test workflow verification and isolation checks
- Expand test-changed coverage and shard selection assertions for CI workflows
- Improve vitest worker temp-directory utilities and related core/CLI tests
- Refine test isolation guardrails and runtime ignore handling for live .fusion noise
- Update contributing guidance and root test script usage for the verified workflow

Fusion-Task-Id: FN-3607
2026-05-06 11:51:48 -07:00
Fusion
7d02ac81ef feat(FN-3593): add test isolation CI enforcement, fix stuck-requeue race, a
This merge lands five FN-3593 commits establishing a test isolation contract with a new `scripts/check-test-isolation.mjs` guard that scans for accidental `beforeEach`/`afterEach`/`beforeAll`/`afterAll` in setup helpers, plus per-package `setup-test-isolation.ts` bootstraps that canonicalize the pat

Fusion-Task-Id: FN-3593
2026-05-06 09:43:51 -07:00
Fusion
c9e776cb34 feat(FN-3293): add stabilization docs to test audit report
Documentation for test stabilization was finalized by updating the test audit report with 2 additional lines.

Fusion-Task-Id: FN-3293
2026-05-04 13:17:57 -07:00