Merge pull request #1695 from Runfusion/fix/full-suite-engine-core-hang

fix(ci): raise shard watchdog floor to 15min to stop Full Suite false-kills
This commit is contained in:
gsxdsm
2026-06-20 22:01:03 -07:00
committed by GitHub
18 changed files with 477 additions and 18 deletions

View File

@@ -0,0 +1,5 @@
---
"@runfusion/fusion": patch
---
Prevent bundled Droid and Claude CLI auth/presence probes from surfacing unhandled promise rejections when `spawn` throws synchronously, such as when test guards block real AI CLI auth commands. These probes now resolve as unavailable/unauthenticated instead of rejecting from fire-and-forget validation paths.

View File

@@ -0,0 +1,5 @@
---
"@runfusion/fusion": patch
---
Fix the experimental left sidebar Settings button so it remains clear of the fixed executor status footer, and keep project-selector fallback labels readable when translations are incomplete.

View File

@@ -77,7 +77,7 @@ Features:
- Task card header meta badges group priority, fast mode, agent-created provenance, and elapsed/created-time chips into one wrapping row; agent labels prefer `sourceMetadata.agentName` over raw agent IDs
- Column ordering semantics: `todo` mirrors scheduler pickup order (priority descending, then oldest `createdAt`, then task ID); `triage`, `in-progress`, `in-review`, and `archived` remain priority-first with task-ID tie-breaks; `done` is ordered by most recent completion first (`columnMovedAt`, then `updatedAt`, then `createdAt` fallback)
- On mobile, both default and workflow-mode boards fill the project viewport while the column strip remains the internal horizontal scroller with contained edge overscroll.
- Board and List workflow switchers use a themed dropdown instead of a native select. The closed trigger shows the workflow name and chevron only; compact Todo / In Progress / Done counts derived from workflow column flags (excluding archived columns) appear while the dropdown is expanded, including on each workflow option.
- Board and List workflow switchers use a themed dropdown instead of a native select. The closed trigger shows the workflow name and chevron only; compact Todo / In Progress / Done counts derived from workflow column flags (excluding archived columns) appear while the dropdown is expanded, including on each workflow option. Those inline count badges intentionally use the same board column color tokens as cards: `--todo`, `--in-progress`, and `--done`.
- When workflow columns are enabled, Board and List hydrate the last successful workflow-lane payload from a per-project session cache; cold loads show a neutral skeleton until settings and workflow metadata are known, avoiding a legacy single-lane flash.
![Board view](./screenshots/dashboard-overview.png)

View File

@@ -0,0 +1,256 @@
---
title: "fix: Resolve Full Suite hang in @fusion/engine and @fusion/core test groups"
type: fix
status: active
date: 2026-06-21
plan_depth: standard
---
# fix: Resolve Full Suite hang in @fusion/engine and @fusion/core test groups
## Summary
The **Full Suite (non-blocking)** workflow on `main` has been red for 30+ consecutive
runs. The failure is **not** a test assertion — every test that reports a result
passes. Instead, three sharded vitest groups never exit and are SIGKILLed by the
CI wall-clock watchdog when they hit their per-group budget:
| Shard | Group | Watchdog budget | Outcome |
|-------|-------|-----------------|---------|
| 1/4 | `@fusion/engine [1/2]` | 405s | killed at budget → shard fails |
| 2/4 | `@fusion/engine [2/2]` | 405s | killed at budget → shard fails |
| 4/4 | `@fusion/core [2/2]` | 338s | killed at budget → shard fails |
| 3/4 | `@fusion/core [1/2]` | 338s | **passes** (32 heartbeats, then exits) |
The `[1/2]` halves and shard 3 finish well under budget, so this is a **genuine
hang in specific tests** — almost certainly leaked open handles (unterminated git
child processes, intervals/timers, or undrained pools) that prevent vitest from
exiting after the test bodies complete — not uniform slowness. The engine tests
that perform real git worktree/branch operations (`Preparing worktree (new branch
'fusion/fn-001')`, repeated `Switched to branch 'main'`) are the leading suspects.
This plan reproduces the hang locally with hanging-process detection, isolates the
leaking test(s), repairs the leak at its source (teardown / process termination /
timer cleanup), and verifies each group exits cleanly within budget.
---
## Problem Frame
- **Authoritative signal:** CI, not a local run. Reference run `27893078004`
(`gh run view 27893078004`), jobs: `Test shard 1/4`, `2/4`, `4/4` failed; `3/4`,
`Engine slow tier`, `Dashboard curated-gate guard` passed.
- **Symptom:** vitest process for a group keeps emitting work (git worktree ops)
and the watchdog `still running` heartbeat, never reaching its `Test Files …`
summary line, until SIGTERM/SIGKILL at budget. A killed group exits non-zero →
the shard fails → the Full Suite workflow fails.
- **Why it matters:** Full Suite is the broadest regression net. While marked
"non-blocking," a permanently-red suite means real regressions in engine/core
can land unnoticed. The two known pre-existing
`shared-branch-group-entry-points.test.ts` failures (see memory) may be related
but are a separate, smaller issue — this plan targets the *hang*.
- **Not in scope:** the `pi-claude-cli`/`droid-cli` "Failed to parse NDJSON line:
{bad" and `packages/cli` "AI metadata generation failed; using fallback" lines —
those are deliberate negative-path test fixtures on passing groups, not failures.
---
## Scope Boundaries
**In scope**
- Diagnose and fix whatever prevents `@fusion/engine` (both splits) and
`@fusion/core [2/2]` from exiting within their watchdog budgets.
- Restore the Full Suite workflow to green on `main`.
**Out of scope / non-goals**
- Rewriting the CI sharding or watchdog-budget derivation logic
(`scripts/ci-test-shard.mjs`, `scripts/lib/run-vitest-watchdog.mjs`). Only touch
budget as a deliberate, justified last resort (see U3 / KTD-2).
- Converting Full Suite to a blocking gate.
### Deferred to Follow-Up Work
- The 2 known `shared-branch-group-entry-points.test.ts` per-task-derivation
failures, unless reproduction shows they are the hang source.
- Broad test-suite speedups beyond what's needed to clear the budget.
---
## Key Technical Decisions
**KTD-1 — Fix the leak, do not raise the budget by default.**
The `[1/2]`/shard-3 halves prove the work fits comfortably under budget, so the
`[2/2]`/engine halves are hanging, not merely slow. Raising the watchdog budget
would mask a real open-handle leak and slow every CI run. Default posture:
identify the leaking test and repair its teardown so vitest exits cleanly.
**KTD-2 — Budget change only with evidence.**
If reproduction proves a group is *legitimately* slow (all tests complete, vitest
exits, but wall-clock genuinely exceeds budget), then and only then adjust the
group's budget/split via `scripts/ci-test-shard.mjs` timings, with the measured
numbers recorded in the commit. This is the documented exception to "don't touch
sharding."
**KTD-3 — Reproduce with the real CI invocation.**
Run the exact per-group vitest command the shard script issues (same
`--project`, same env) under a wall-clock `timeout`, plus vitest's
hanging-process reporter, so local results match CI rather than the cached
`pnpm test` changed-file path.
---
## High-Level Technical Design
```
CI shard → pnpm --filter @fusion/engine exec vitest run --project=…
│
├── all test bodies pass ✅ (Test Files summary never printed)
│
└── process does NOT exit ❌
│ leaked handle keeps event loop alive:
│ • spawned git/worktree child process not awaited/killed
│ • setInterval / heartbeat timer not cleared in teardown
│ • worktree-pool / db handle not closed
▼
watchdog budget reached → SIGTERM/SIGKILL → exit≠0 → shard FAIL
```
Fix target = close the leaked handle in `afterEach`/`afterAll` (or in the
production code path the test exercises) so the process exits naturally.
---
## Implementation Units
### U1. Reproduce the hang locally and isolate the leaking test(s)
**Goal:** Turn the CI-only failure into a deterministic local repro and name the
exact test file(s) and handle that keep the process alive.
**Dependencies:** none.
**Files (investigation, no production edits yet):**
- `packages/engine/vitest.config.ts` (projects: `engine-default`,
`engine-reliability`, `engine-core`, `engine-slow`)
- `scripts/ci-test-shard.mjs`, `scripts/lib/run-vitest-watchdog.mjs` (read-only:
confirm the exact per-group command + budget)
- Suspect real-git/worktree tests under `packages/engine/src/__tests__/`:
`worktree-acquisition.test.ts`, `worktree-pool-liveness.test.ts`,
`merger-integration-worktree.test.ts`,
`merger-finalize-unproven.real-git.test.ts`,
`self-healing-orphan-only-scope.real-git.test.ts`,
`self-healing-ghost-branch-recovery.test.ts`, `executor-worktree.test.ts`,
`restart.integration.test.ts`, `run-audit.integration.test.ts`
**Approach:**
1. Derive the exact group command from `ci-test-shard.mjs` for `engine [1/2]`,
`engine [2/2]`, and `core [2/2]`.
2. Run each under a hard wall-clock `timeout` (e.g. 420s) with hanging-process
detection — vitest `--reporter=hanging-process` (or `--reporter=verbose
--no-file-parallelism` plus `why-is-node-running`-style logging) to print the
handles still open after the run finishes.
3. Confirm the signature: "Test Files … passed" never prints (or prints but
process doesn't exit) and the reporter names the dangling handle/test file.
4. Record the offending file(s) + handle type for U2.
**Execution note:** Characterization-first — establish the failing repro and
capture the open-handle report *before* changing any production or test code.
Respect existing guards: do not kill the live dashboard port (port-4040 guards,
`FUSION_RESERVED_PORTS`) and do not kill the running dev instance.
**Test scenarios:** Test expectation: none — this unit is diagnostic; it produces
a repro recipe and a named culprit, not new tests.
**Verification:** A documented command that reliably hangs/leaks locally, and a
hanging-process report naming the test file(s) and handle keeping the loop alive.
---
### U2. Fix the leaked handle so the group exits cleanly
**Goal:** Eliminate the dangling handle so vitest exits on its own for all three
groups, with zero behavior change to the code under test.
**Dependencies:** U1.
**Files:** the test file(s) and/or production module(s) identified in U1.
Likely candidates (confirm in U1, do not assume): worktree-pool / git child
process spawn sites and their `afterEach`/`afterAll` teardown; any
`setInterval`/heartbeat timer (e.g. liveness/heartbeat code) not cleared on
teardown; db/sqlite handles left open.
**Approach (apply whichever U1 proves):**
- Await and/or terminate spawned git/worktree child processes in teardown; ensure
no detached process outlives the test.
- Clear timers/intervals registered by the code under test (use fake timers or an
explicit `clearInterval` in teardown).
- Close pools/db handles opened by the test.
- Prefer fixing the leak at the production source if the same handle could leak in
real runtime; otherwise fix the test's teardown.
**Patterns to follow:** Mirror teardown patterns in the engine `[1/2]`/core
`[1/2]` tests that already exit cleanly. Reuse existing temp-dir/worktree cleanup
helpers in `packages/engine/src/__tests__/`.
**Test scenarios:**
- Happy path: the previously-hanging test still asserts its original behavior and
passes.
- Resource cleanup: after the test, the hanging-process reporter shows **no**
dangling handle for the fixed file (regression guard against re-introduction).
- Edge: if a child process is killed in teardown, a test where the process already
exited does not throw on double-kill.
**Verification:** Each group runs to its `Test Files … passed` summary AND the
process exits 0 without watchdog intervention.
---
### U3. Verify all three groups finish within budget and Full Suite goes green
**Goal:** Confirm the fix across `engine [1/2]`, `engine [2/2]`, and
`core [2/2]`, and that the workflow passes end-to-end.
**Dependencies:** U2.
**Files:** none (verification). Only touch `scripts/ci-test-shard.mjs` timings if
KTD-2's slow-not-hung condition is proven in U1/U2.
**Approach:**
1. Re-run each group's exact command under the same wall-clock `timeout`; confirm
each exits 0 comfortably under its watchdog budget (405s / 405s / 338s).
2. Push the branch (or trigger the Full Suite via `workflow_dispatch`) and watch
the four shards go green via `gh pr checks --watch` / `gh run watch`.
3. If a group is proven slow-not-hung, apply the KTD-2 budget/split adjustment
with measured numbers in the commit message.
**Test scenarios:** Test expectation: none — verification unit.
**Verification:** Full Suite (non-blocking) run on the branch reports all shards
`success`; no group is SIGKILLed at budget.
---
## Risks & Dependencies
- **Repro may be environment-sensitive.** If the hang is CI-specific (e.g. git
identity, worktree path layout, missing TTY), local repro in U1 may not trigger
it. Mitigation: replicate CI env vars from the shard script; if still not
reproducible, drive the diagnosis from a `workflow_dispatch` run with the
hanging-process reporter enabled and artifacts uploaded.
- **vitest auto-kill history (memory):** older fn TUI builds SIGKILLed `vitest`
processes every 30s. Ensure the TUI is not running the broken build during
local repro, or run the repro outside the TUI, so a clean exit isn't mistaken
for a kill.
- **Multiple independent leaks.** engine `[1/2]` and `[2/2]` both failing may mean
more than one leaking test; treat U1/U2 as iterative until all three groups are
green.
## Sources & Research
- CI run `27893078004` (jobs + full log) — failure signature, watchdog budgets,
per-group heartbeat counts.
- `gh run list --branch main --workflow "Full Suite (non-blocking)"` — 30+
consecutive failures (chronic, not a fresh regression).
- `packages/engine/vitest.config.ts` — 30s testTimeout, 45s hookTimeout, project
splits.
- Memory: "Branch-group known failures", "vitest auto-kill incident",
"Port 4040 kill guards", "Engine src has no tsc emit".

View File

@@ -1036,7 +1036,8 @@ function AppInner() {
Experimental left sidebar navigation replaces the Header view shortcuts with a persistent sidebar on non-mobile project screens, while mobile continues to use the bottom navigation bar as the only primary navigation surface.
*/
const leftSidebarNavEnabled = experimentalFeatures.leftSidebarNav === true;
const sidebarActive = leftSidebarNavEnabled && !isMobile && viewMode === "project" && !!currentProject;
const executorFooterVisible = viewMode === "project" && !!currentProject;
const sidebarActive = leftSidebarNavEnabled && !isMobile && executorFooterVisible;
const agentOnboardingEnabled = experimentalFeatures.agentOnboarding === true;
const agentsEnabled = true;
@@ -2136,15 +2137,16 @@ function AppInner() {
currentProject={currentProject}
onSelectProject={handleSelectProject}
onViewAllProjects={handleViewAllProjects}
footerVisible={executorFooterVisible}
/>
)}
<div
className={`project-content${viewMode === "project" && currentProject && (!isMobile || !mobileKeyboardOpen) ? " project-content--with-footer" : ""}${isMobile && !mobileKeyboardOpen ? " project-content--with-mobile-nav" : ""}`}
className={`project-content${executorFooterVisible && (!isMobile || !mobileKeyboardOpen) ? " project-content--with-footer" : ""}${isMobile && !mobileKeyboardOpen ? " project-content--with-mobile-nav" : ""}`}
>
{renderMainContent()}
</div>
</div>
{viewMode === "project" && currentProject && (
{executorFooterVisible && currentProject && (
<ExecutorStatusBar
tasks={isRemote && remoteData.tasks.length > 0 ? remoteData.tasks : tasks}
projectId={currentProject.id}

View File

@@ -8,6 +8,7 @@ The experimental sidebar is a persistent desktop/tablet navigation replacement f
position: relative;
display: flex;
flex-direction: column;
box-sizing: border-box;
width: var(--left-sidebar-nav-width);
min-width: var(--left-sidebar-nav-width);
min-height: 0;
@@ -16,6 +17,14 @@ The experimental sidebar is a persistent desktop/tablet navigation replacement f
color: var(--text);
}
/*
FNXC:Navigation 2026-06-20-00:00:
The left sidebar is a sibling of project-content, so it does not inherit project-content footer padding. When the fixed executor status bar is rendered, reserve the shared --executor-footer-height on the aside so the bottom Settings button remains visible and clickable above the footer.
*/
.left-sidebar-nav--with-footer {
padding-bottom: var(--executor-footer-height);
}
.left-sidebar-nav__collapse-toggle {
flex-shrink: 0;
}

View File

@@ -115,6 +115,7 @@ export interface LeftSidebarNavProps {
currentProject?: ProjectInfo | null;
onSelectProject?: (project: ProjectInfo) => void;
onViewAllProjects?: () => void;
footerVisible?: boolean;
}
function formatCount(count: number): string {
@@ -160,6 +161,7 @@ export function LeftSidebarNav({
pluginDashboardViews = [],
showAgentsTab = false,
showSkillsTab = false,
footerVisible = false,
}: LeftSidebarNavProps) {
const { t } = useTranslation("app");
const [sidebarWidth, setSidebarWidth] = useState(readStoredSidebarWidth);
@@ -390,7 +392,7 @@ export function LeftSidebarNav({
return (
<aside
className={`left-sidebar-nav${isCollapsed ? " left-sidebar-nav--collapsed" : ""}`}
className={`left-sidebar-nav${isCollapsed ? " left-sidebar-nav--collapsed" : ""}${footerVisible ? " left-sidebar-nav--with-footer" : ""}`}
data-testid="left-sidebar-nav"
aria-label={t("nav.sidebarAriaLabel", "Sidebar navigation")}
style={isCollapsed ? undefined : { width: sidebarWidth, minWidth: sidebarWidth }}

View File

@@ -580,6 +580,7 @@
* On mobile the token is overridden to 32px to match the shorter footer.
*/
.dashboard-project-shell {
--executor-footer-height: 36px;
display: flex;
flex: 1;
min-height: 0;
@@ -602,7 +603,6 @@
}
.project-content--with-footer {
--executor-footer-height: 36px;
padding-bottom: var(--executor-footer-height);
}

View File

@@ -83,6 +83,11 @@ export function ProjectSelector({
viewAllLabel,
}: ProjectSelectorProps) {
const { t } = useTranslation("app");
/*
* FNXC:ProjectSelector 2026-06-20-20:51:
* The trigger fallback must remain readable when translation fixtures omit the optional project-selector key; provide the English default at the component seam instead of leaking the i18n key into the header.
*/
const projectsLabel = t("projectSelector.projects", "Projects");
const [isOpen, setIsOpen] = useState(false);
const [searchQuery, setSearchQuery] = useState("");
const [highlightedIndex, setHighlightedIndex] = useState(-1);
@@ -389,12 +394,12 @@ export function ProjectSelector({
aria-expanded={isOpen}
aria-haspopup="listbox"
aria-label={t("projectSelector.ariaLabel", "Select project")}
title={currentProject?.name ? t("projectSelector.switchProjectTitle", "Switch project (current: {{name}})", { name: currentProject.name }) : t("projectSelector.projectsTitle")}
title={currentProject?.name ? t("projectSelector.switchProjectTitle", "Switch project (current: {{name}})", { name: currentProject.name }) : t("projectSelector.projectsTitle", "Projects")}
data-testid="project-selector-trigger"
>
<Folder size={16} className="project-selector__trigger-icon" />
<span className="project-selector__trigger-text">
{currentProject?.name || t("projectSelector.projects")}
{currentProject?.name || projectsLabel}
</span>
<ChevronDown
size={14}
@@ -407,7 +412,7 @@ export function ProjectSelector({
<div
className="project-selector__dropdown"
role="listbox"
aria-label={t("projectSelector.projects")}
aria-label={projectsLabel}
onKeyDown={handleDropdownKeyDown}
data-testid="project-selector-dropdown"
>

View File

@@ -91,16 +91,20 @@
line-height: calc(var(--space-md) / var(--space-sm));
}
/*
FNXC:WorkflowSwitcher 2026-06-20-00:00:
The switcher's inline Todo, In Progress, and Done count badges intentionally mirror the board column color tokens so each count reads as the same color as the column it summarizes.
*/
.workflow-switcher-count--todo {
color: var(--text-muted);
color: var(--todo);
}
.workflow-switcher-count--in-progress {
color: var(--color-warning);
color: var(--in-progress);
}
.workflow-switcher-count--done {
color: var(--color-success);
color: var(--done);
}
.workflow-switcher-count-separator {

View File

@@ -57,6 +57,12 @@ function expectNoSidebarBrandOrProjectAffordances(container: HTMLElement) {
expect(container.querySelector(".left-sidebar-nav__wordmark")).toBeNull();
}
function expectSettingsLastInFooter() {
const footer = screen.getByTestId("sidebar-nav-settings").closest(".left-sidebar-nav__footer");
expect(footer).not.toBeNull();
expect(footer?.lastElementChild).toBe(screen.getByTestId("sidebar-nav-settings"));
}
function renderSidebar(overrides: Partial<ComponentProps<typeof LeftSidebarNav>> = {}) {
const onChangeView = vi.fn();
const props: ComponentProps<typeof LeftSidebarNav> = {
@@ -127,6 +133,36 @@ describe("LeftSidebarNav", () => {
expect(sidebarButtons.at(-1)).toBe(screen.getByTestId("sidebar-nav-settings"));
});
it.each([
["expanded", false],
["collapsed", true],
])("applies footer clearance only when the executor footer is visible in %s mode", (_label, collapsed) => {
if (collapsed) {
window.localStorage.setItem("fusion:left-sidebar-collapsed", "true");
}
const withFooter = renderSidebar({ footerVisible: true });
const sidebarWithFooter = screen.getByTestId("left-sidebar-nav");
expect(sidebarWithFooter).toHaveClass("left-sidebar-nav--with-footer");
if (collapsed) {
expect(sidebarWithFooter).toHaveClass("left-sidebar-nav--collapsed");
}
expectSettingsLastInFooter();
withFooter.unmount();
if (collapsed) {
window.localStorage.setItem("fusion:left-sidebar-collapsed", "true");
}
renderSidebar();
const sidebarWithoutFooter = screen.getByTestId("left-sidebar-nav");
expect(sidebarWithoutFooter).not.toHaveClass("left-sidebar-nav--with-footer");
if (collapsed) {
expect(sidebarWithoutFooter).toHaveClass("left-sidebar-nav--collapsed");
}
expectSettingsLastInFooter();
});
it("gates optional destinations on their matching feature flags and props while preserving bottom settings", () => {
renderSidebar({
showAgentsTab: false,

View File

@@ -1,6 +1,7 @@
import { fireEvent, render, screen, within } from "@testing-library/react";
import { describe, expect, it, vi } from "vitest";
import type { BoardWorkflowDefinition } from "../../api";
import { loadAllAppCssBaseOnly } from "../../test/cssFixture";
import { WorkflowSwitcher } from "../WorkflowSwitcher";
import type { WorkflowStatusCounts } from "../workflowStatusCounts";
@@ -21,6 +22,11 @@ function countMap(entries: Array<[string, WorkflowStatusCounts]> = []) {
return new Map<string, WorkflowStatusCounts>(entries);
}
function cssRuleFor(css: string, selector: string) {
const escaped = selector.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
return css.match(new RegExp(`${escaped}\\s*\\{([^}]*)\\}`))?.[1] ?? "";
}
describe("WorkflowSwitcher", () => {
it("renders the active workflow without compact counts while collapsed", () => {
render(
@@ -109,4 +115,20 @@ describe("WorkflowSwitcher", () => {
expect(within(designOption).getByText("0", { selector: ".workflow-switcher-count--in-progress" })).toBeInTheDocument();
expect(within(designOption).getByText("0", { selector: ".workflow-switcher-count--done" })).toBeInTheDocument();
});
it("colors status counts with board column color tokens", () => {
const css = loadAllAppCssBaseOnly();
const badgeRules = [
[".workflow-switcher-count--todo", "--todo"],
[".workflow-switcher-count--in-progress", "--in-progress"],
[".workflow-switcher-count--done", "--done"],
] as const;
for (const [selector, token] of badgeRules) {
const rule = cssRuleFor(css, selector);
expect(rule).toMatch(new RegExp(`color:\\s*var\\(${token}\\)`));
expect(rule).not.toMatch(/var\(--(?:text-muted|color-warning|color-success)\)/);
expect(rule).not.toMatch(/#[0-9a-fA-F]{3,8}|rgba?\(/);
}
});
});

View File

@@ -415,6 +415,16 @@ describe("validateCliPresenceAsync", () => {
const result = await validateCliPresenceAsync();
expect(result.ok).toBe(false);
});
it("resolves ok=false instead of rejecting when droid spawn throws synchronously", async () => {
(spawn as any).mockImplementationOnce(() => {
throw new Error("Real AI CLI launch blocked during tests: droid --version");
});
await expect(validateCliPresenceAsync()).resolves.toMatchObject({
ok: false,
});
});
});
describe("validateCliAuthAsync", () => {
@@ -452,6 +462,21 @@ describe("validateCliAuthAsync", () => {
);
warnSpy.mockRestore();
});
it("resolves false instead of rejecting when droid auth spawn throws synchronously", async () => {
(spawn as any).mockImplementationOnce(() => {
throw new Error(
"Real AI CLI launch blocked during tests: droid auth status",
);
});
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
await expect(validateCliAuthAsync()).resolves.toBe(false);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining("not authenticated"),
);
warnSpy.mockRestore();
});
});
describe("CLI flags", () => {

View File

@@ -414,6 +414,16 @@ describe("validateCliPresenceAsync", () => {
const result = await validateCliPresenceAsync();
expect(result.ok).toBe(false);
});
it("resolves ok=false instead of rejecting when claude spawn throws synchronously", async () => {
(spawn as any).mockImplementationOnce(() => {
throw new Error("Real AI CLI launch blocked during tests: claude --version");
});
await expect(validateCliPresenceAsync()).resolves.toMatchObject({
ok: false,
});
});
});
describe("validateCliAuthAsync", () => {
@@ -451,6 +461,21 @@ describe("validateCliAuthAsync", () => {
);
warnSpy.mockRestore();
});
it("resolves false instead of rejecting when claude auth spawn throws synchronously", async () => {
(spawn as any).mockImplementationOnce(() => {
throw new Error(
"Real AI CLI launch blocked during tests: claude auth status",
);
});
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
await expect(validateCliAuthAsync()).resolves.toBe(false);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining("not authenticated"),
);
warnSpy.mockRestore();
});
});
describe("CLI flags", () => {

View File

@@ -219,10 +219,20 @@ export function captureStderr(proc: ChildProcess): () => string {
* does this on every chat send), sync probes freeze every other request.
* This async variant uses spawn so the loop keeps turning while the subprocess
* starts up.
*
* FNXC:CliRuntime 2026-06-20-17:25:
* FN-6808/FN-6801 require this fire-and-forget auth/presence probe to never reject. Catch synchronous spawn throws from the Vitest child-process guard or platform launch errors and resolve 127, matching the async error sentinel so callers degrade to unauthenticated/not-present instead of surfacing unhandled promise rejections.
*/
function runClaudeProbe(args: string[], timeoutMs = 5000): Promise<number> {
return new Promise((resolve) => {
const proc = spawn("claude", args, { stdio: "ignore" });
let proc: ChildProcess;
try {
proc = spawn("claude", args, { stdio: "ignore" });
} catch {
resolve(127);
return;
}
const timer = setTimeout(() => {
try {
proc.kill("SIGKILL");

View File

@@ -219,10 +219,20 @@ export function captureStderr(proc: ChildProcess): () => string {
* does this on every chat send), sync probes freeze every other request.
* This async variant uses spawn so the loop keeps turning while the subprocess
* starts up.
*
* FNXC:CliRuntime 2026-06-20-17:25:
* FN-6808/FN-6801 require this fire-and-forget auth/presence probe to never reject. Catch synchronous spawn throws from the Vitest child-process guard or platform launch errors and resolve 127, matching the async error sentinel so callers degrade to unauthenticated/not-present instead of surfacing unhandled promise rejections.
*/
function runDroidProbe(args: string[], timeoutMs = 45000): Promise<number> {
return new Promise((resolve) => {
const proc = spawn("droid", args, { stdio: "ignore" });
let proc: ChildProcess;
try {
proc = spawn("droid", args, { stdio: "ignore" });
} catch {
resolve(127);
return;
}
const timer = setTimeout(() => {
try {
proc.kill("SIGKILL");
@@ -275,11 +285,22 @@ export async function validateCliAuthAsync(): Promise<boolean> {
}
export async function discoverDroidModels(): Promise<string[]> {
const attempts: string[][] = [["models", "--json"], ["model", "list", "--json"], ["models"]];
const attempts: string[][] = [
["models", "--json"],
["model", "list", "--json"],
["models"],
];
for (const args of attempts) {
const models = await new Promise<string[] | null>((resolve) => {
const proc = spawn("droid", args, { stdio: ["ignore", "pipe", "ignore"] });
let proc: ChildProcess;
try {
proc = spawn("droid", args, { stdio: ["ignore", "pipe", "ignore"] });
} catch {
resolve(null);
return;
}
let out = "";
proc.stdout?.on("data", (chunk: Buffer) => {
out += chunk.toString();

View File

@@ -39,7 +39,9 @@ test("deriveBudgetMs: no fresh timing falls back to the per-class ceiling", () =
test("deriveBudgetMs: fresh timing tightens within the band", () => {
// expected×multiplier between floor and ceiling → use the tightened value.
const expected = 200_000; // 200s
// 300s × 3.5 = 1050s, which sits between the shard floor (15min) and
// ceiling (30min) so the tightened value is used un-clamped.
const expected = 300_000; // 300s
const derived = deriveBudgetMs({ klass: "shard", expectedDurationMs: expected, timingsFresh: true });
assert.equal(derived, Math.round(expected * DEFAULT_BUDGET_MULTIPLIER));
assert.ok(derived >= CLASS_BUDGET_BANDS.shard.floor);
@@ -59,6 +61,21 @@ test("deriveBudgetMs: clamps to floor and ceiling", () => {
);
});
test("deriveBudgetMs: shard floor pins heavy slices above the false-kill window", () => {
// FNXC:TestInfrastructure 2026-06-20-21:52:
// Regression guard for the 5min -> 15min shard-floor raise. A value whose
// expected×multiplier lands in the *old* un-clamped window (300s..900s) must
// now clamp UP to the 15min floor. 150s × 3.5 = 525s, which was returned
// verbatim under the old 5min floor but is below the new one. Pinning the
// concrete floor value here means an accidental revert to 5min fails loudly
// instead of silently re-tightening the engine/core slices into SIGKILLs.
assert.equal(CLASS_BUDGET_BANDS.shard.floor, 15 * 60_000);
assert.equal(
deriveBudgetMs({ klass: "shard", expectedDurationMs: 150_000, timingsFresh: true }),
CLASS_BUDGET_BANDS.shard.floor,
);
});
test("deriveBudgetMs: unknown class falls back to the changed band", () => {
assert.equal(deriveBudgetMs({ klass: "nonexistent" }), CLASS_BUDGET_BANDS.changed.ceiling);
});

View File

@@ -36,7 +36,22 @@ const MINUTE = 60_000;
*/
export const CLASS_BUDGET_BANDS = {
// One CI shard command (may fan out across several packages via --filter).
shard: { floor: 5 * MINUTE, ceiling: 30 * MINUTE },
/*
FNXC:TestInfrastructure 2026-06-20-21:51:
The shard watchdog floor is 15min, not 5min. The heaviest CI shard slices
(@fusion/engine and @fusion/core, each split [1/2]+[2/2]) are import- and
real-git-subprocess heavy, and the committed scripts/test-timings.json
undercounts that overhead (most files bucket to the 100ms floor, so the
snapshot total sits far below current wall-clock). A "fresh" but undercounting
snapshot was tightening these slices to ~340-405s and SIGKILLing healthy runs
on slower CI runners (engine[2/2]=145s, core[2/2]=283s locally, both pass) —
the exact false-kill the floor/ceiling band exists to prevent (see the
deriveBudgetMs note above and plan KTD-2). 15min sits above the observed CI
need while still bounding a true hang far under the job's 60min ceiling.
The dashboard-lane floor is also 15min but for separate historical reasons;
the two values are not coupled and may diverge.
*/
shard: { floor: 15 * MINUTE, ceiling: 30 * MINUTE },
// One local changed-file package invocation.
changed: { floor: 2 * MINUTE, ceiling: 20 * MINUTE },
// One dashboard quality lane (heap-managed). Matches the historical 15min.