FN-9139: add bounded PostgreSQL pre-admission measurement
Establish an inert PostgreSQL setup signal and repeatable evidence tooling without changing harness behavior. - add explicit setup participation semantics and inertness coverage - survey Vitest setup boundaries with isolated report-only fixtures - bound interleaved campaigns by process group and campaign deadline - reject missing backend samples and enable candidate diagnostics - document the rejected boundary result and successor protocol Files changed: .../test-failures/postgres-ddl-admission-bound.md | 15 ++ docs/testing.md | 10 ++ packages/core/package.json | 2 +- .../src/__test-utils__/pg-setup-participation.ts | 22 +++ .../src/__tests__/pg-setup-participation.test.ts | 21 +++ .../__tests__/vitest-setup-pg-inertness.test.ts | 10 ++ packages/core/vitest.pg.config.ts | 10 ++ .../__tests__/pg-preadmission-campaign.test.mjs | 63 +++++++ scripts/__tests__/pg-setup-boundary-probe.test.mjs | 92 ++++++++++ scripts/pg-preadmission-campaign.mjs | 194 +++++++++++++++++++++ scripts/pg-setup-boundary-probe.mjs | 182 +++++++++++++++++++ 11 files changed, 620 insertions(+), 1 deletion(-) Fusion-Task-Id: FN-9139 Fusion-Task-Lineage: 2cf8ccbf-37f9-4fdc-8e8f-326df823e1cd Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
gsxdsm
@@ -33,6 +33,21 @@ FN-9130 then implemented bounded deferred draining (R=2, Q=8), per-file flush, a
|
||||
|
||||
Candidate C, per-fork database reuse with `TRUNCATE`, remains unexplored and is filed as FN-9136. Lowering workers, widening timeouts, drop retries, and core-test quarantine were not remedies and were not used.
|
||||
|
||||
## FN-9139 setup-boundary pre-admission result — rejected
|
||||
|
||||
<!-- FNXC:PgTestPreAdmission 2026-08-17-03:20: A shared Vitest setup boundary may only carry PostgreSQL work after a deterministic survey proves its timeout ownership. FN-9139 rejects uncertainty rather than attaching another acquisition to a hook budget. -->
|
||||
|
||||
FN-9139 recorded a reachable local cluster (`max_connections=100`, `superuser_reserved_connections=3`, 28 CPUs) and a 27-worker control run. Its report-only boundary fixture ran twice: global setup appeared once per invocation and setup-file top-level evaluation once per worker, but the controlled short-deadline run yielded `unknown-failure` timeout classification rather than a stable off-budget proof. The required candidate probe was therefore deleted/not retained before a loaded candidate campaign could be started. This is an `insufficient-data` **BOUNDARY REJECTED** result, not a repaired timeout and not a reason to change retries, timeouts, harness wiring, or worker caps.
|
||||
|
||||
| Arm / run | Wall time | Failed files | `project-identity.test.ts:41:3` | Peak `pg_stat_activity` backends | Probe degradation |
|
||||
| --- | ---: | ---: | --- | --- | --- |
|
||||
| Control / Step 0 | 218.8s | 16 | unavailable — the Step-0 summary did not retain case-level output | unavailable — the campaign sampler had not yet been implemented | n/a (probe off) |
|
||||
| Candidate | not run — boundary proof rejected before probe retention | n/a | n/a | n/a | n/a |
|
||||
|
||||
The unavailable Step-0 fields are deliberately not backfilled or inferred: no sampler JSONL or captured Vitest log survives from that pre-driver control run. The campaign protocol requires five valid interleaved samples per arm with candidate diagnostics, so this single control observation cannot be promoted into affordability evidence.
|
||||
|
||||
The retained `pg-setup-participation.ts` signal is connectionless and explicit (`FUSION_PG_TEST_SETUP_PARTICIPANT=1`, overridden by `FUSION_PG_TEST_SKIP=1`), preserving non-PG setup inertness. `scripts/pg-setup-boundary-probe.mjs` and `scripts/pg-preadmission-campaign.mjs` remain as repeatable measurement tooling. Any successor must first prove ordering and off-budget ownership, keep the probe default-off and non-consumable by the harness, use interleaved one-shell measurements, and treat incomplete data as rejection.
|
||||
|
||||
<!-- FNXC:PgDdlLaneMetric 2026-08-17-00:59: FN-9134 must establish a green, drift-resistant control band before another structural DDL candidate is allowed to claim improvement. An invalid control is terminal insufficient data, not a reason to tune timeouts or rerun unfavorable samples. -->
|
||||
|
||||
<!-- FNXC:PgDdlLaneMetric 2026-08-17-02:55: The required seven-pair interleaved campaign must remain the terminal evidence. Zeroed inline watchdogs only reflect off-hook execution, while green lanes and zero survivors decide whether a candidate can ship. -->
|
||||
|
||||
@@ -31,6 +31,16 @@ FN-9131 retains `pg-connection-budget.ts` as a tested cluster-shared advisory-lo
|
||||
|
||||
The initial harness wiring made a 27-worker PostgreSQL directory run worse, even after registry contention was changed from bounded rejection to queueing. Do not restore the wiring, alter test timeouts, add retries, or cap workers as a workaround. FN-9139 owns selecting a per-worker lifecycle point that is inert for non-PostgreSQL lanes and can charge admission wait outside individual test budgets. See [the terminal-negative record](solutions/test-failures/pg-harness-connection-budget.md).
|
||||
|
||||
### PostgreSQL setup-boundary participation and measurement
|
||||
|
||||
<!-- FNXC:PgTestPreAdmission 2026-08-17-03:20: FN-9139 requires the shared Vitest setup path to remain connectionless for every non-PostgreSQL lane. Participation is an explicit environment signal, rather than a reachability probe, because importing the harness itself performs a TCP probe. -->
|
||||
|
||||
`FUSION_PG_TEST_SETUP_PARTICIPANT=1` is the only opt-in for a PostgreSQL setup participant. It is intentionally accepted from a bare exported environment variable as well as the dedicated PostgreSQL configuration, so an ad-hoc directory reproduction has the same participation semantics. Any absent, malformed, empty, whitespace, or `0` value is non-participating; `FUSION_PG_TEST_SKIP=1` overrides participation. Setup work on the shared file must leave non-participating lanes with no PostgreSQL import, socket, filesystem token, or environment mutation.
|
||||
|
||||
`FUSION_PG_TEST_PREADMISSION_PROBE=1` is reserved for a default-off, observation-only measurement probe and is consulted only by a participating lane. A boundary probe must publish no token, lease, registry, singleton, or environment value that the harness can consume: coupling it would be an unmeasured admission rewiring, not a boundary experiment. `FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS=1` enables the existing JSONL measurement channel in both arms.
|
||||
|
||||
Use `scripts/pg-setup-boundary-probe.mjs` to survey setup boundaries before changing a shared setup hook. Its fixture reports boundary granularity and ordering; it never runs a repository suite. The loaded campaign is driven only by `scripts/pg-preadmission-campaign.mjs`, which alternates control and candidate arms, samples one shell's diagnostics, and enforces a five-minute run limit and sixty-minute campaign limit. Collect at least five valid runs per arm. The candidate is affordable only when its median wall time is no worse than the control maximum, its maximum failed-file count is no worse than control, its peak backend footprint is at most one above control, and it records no probe degradation. Fewer than five valid samples, invalid candidate probe records, a capped campaign, or an unavailable environment is deterministically `insufficient-data`, not a reason to tune timeouts or repeat unfavorable results.
|
||||
|
||||
<!-- FNXC:EngineTests 2026-07-08-03:00: FN-7667 decouples the engine-core gate's module graph from full-barrel growth so new feature modules don't silently inflate every gate fork's transform/import cost. -->
|
||||
**Gate-safe `@fusion/core` barrel:** the `engine-core` project resolves `@fusion/core` to `packages/core/src/index.gate.ts` (a project-scoped `resolve.alias`, not the root map), not the full `packages/core/src/index.ts` barrel. `index.gate.ts` is a byte-for-byte copy of the full barrel minus the `export ... from` statements for modules added to the barrel after the last re-audit baseline — i.e. it re-exports everything the full barrel does except genuinely new, gate-irrelevant feature modules (diffed against the prior baseline commit's barrel, not hand-picked from what gate *test* files import — production modules under test pull in far more of the barrel transitively than their own imports suggest). `engine-default`/`engine-reliability`/`engine-slow` are unaffected and keep resolving the full barrel. `@fusion/engine` is untouched (no gate file imports it). When adding a new barrel module that no gate test needs, mirror the exclusion in `index.gate.ts` rather than letting gate wall-time grow — see the FNXC comment at the top of `index.gate.ts` and `packages/engine/vitest.config.ts`'s `engine-core` project for the audit procedure.
|
||||
|
||||
|
||||
@@ -63,7 +63,7 @@
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "vitest run --silent=passed-only --reporter=dot",
|
||||
"test:embedded-postgres": "vitest run src/__tests__/postgres/embedded-lifecycle.test.ts --silent=passed-only --reporter=dot",
|
||||
"test:pg-gate": "vitest run --config vitest.pg.config.ts src/__tests__/postgres/handoff-to-review-atomicity.pg.test.ts src/__tests__/postgres/task-lifecycle-e2e.pg.test.ts --silent=passed-only --reporter=dot",
|
||||
"test:pg-gate": "FUSION_PG_TEST_SETUP_PARTICIPANT=1 vitest run --config vitest.pg.config.ts src/__tests__/postgres/handoff-to-review-atomicity.pg.test.ts src/__tests__/postgres/task-lifecycle-e2e.pg.test.ts --silent=passed-only --reporter=dot",
|
||||
"test:unit-gate": "vitest run src/__tests__/task-merge.test.ts src/__tests__/legacy-adoption.test.ts src/__tests__/no-hardcoded-lifecycle-columns.test.ts src/__tests__/sync-workflow-ir-callsite-allowlist.test.ts --silent=passed-only --reporter=dot"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
22
packages/core/src/__test-utils__/pg-setup-participation.ts
Normal file
22
packages/core/src/__test-utils__/pg-setup-participation.ts
Normal file
@@ -0,0 +1,22 @@
|
||||
export const PG_SETUP_PARTICIPATION_ENV = "FUSION_PG_TEST_SETUP_PARTICIPANT";
|
||||
|
||||
export type PgSetupParticipationReason = "enabled" | "skip-requested" | "not-opted-in";
|
||||
export type PgSetupParticipation =
|
||||
| { participating: true; reason: "enabled" }
|
||||
| { participating: false; reason: Exclude<PgSetupParticipationReason, "enabled"> };
|
||||
|
||||
/**
|
||||
* FNXC:PgTestPreAdmission 2026-08-17-03:20:
|
||||
* FN-9139 requires every package sharing Vitest setup to remain inert unless a
|
||||
* PostgreSQL lane explicitly opts in. This resolver is deliberately pure and
|
||||
* env-resolvable rather than config-only: importing the harness would perform
|
||||
* its TCP reachability probe and tax non-participant lanes.
|
||||
*/
|
||||
export function resolvePgSetupParticipation(
|
||||
env: NodeJS.ProcessEnv = process.env,
|
||||
): PgSetupParticipation {
|
||||
if (env.FUSION_PG_TEST_SKIP === "1") return { participating: false, reason: "skip-requested" };
|
||||
return env[PG_SETUP_PARTICIPATION_ENV] === "1"
|
||||
? { participating: true, reason: "enabled" }
|
||||
: { participating: false, reason: "not-opted-in" };
|
||||
}
|
||||
21
packages/core/src/__tests__/pg-setup-participation.test.ts
Normal file
21
packages/core/src/__tests__/pg-setup-participation.test.ts
Normal file
@@ -0,0 +1,21 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { resolvePgSetupParticipation } from "../__test-utils__/pg-setup-participation.js";
|
||||
|
||||
describe("PostgreSQL setup participation", () => {
|
||||
it.each([undefined, "", " ", "0", "true", "1 "])("keeps %j non-participating", (value) => {
|
||||
expect(resolvePgSetupParticipation(value === undefined ? {} : { FUSION_PG_TEST_SETUP_PARTICIPANT: value })).toEqual({ participating: false, reason: "not-opted-in" });
|
||||
});
|
||||
|
||||
it("accepts only the explicit participant value and honors skip", () => {
|
||||
expect(resolvePgSetupParticipation({ FUSION_PG_TEST_SETUP_PARTICIPANT: "1" })).toEqual({ participating: true, reason: "enabled" });
|
||||
expect(resolvePgSetupParticipation({ FUSION_PG_TEST_SETUP_PARTICIPANT: "1", FUSION_PG_TEST_SKIP: "1", FUSION_PG_TEST_PREADMISSION_PROBE: "1" })).toEqual({ participating: false, reason: "skip-requested" });
|
||||
});
|
||||
|
||||
it("has no PostgreSQL or harness dependency", () => {
|
||||
const source = readFileSync(new URL("../__test-utils__/pg-setup-participation.ts", import.meta.url), "utf8");
|
||||
expect(source).not.toMatch(/^\s*import\s/m);
|
||||
expect(source).not.toMatch(/postgres|pg-test-harness|node:net|node:fs/);
|
||||
expect(source).not.toMatch(/process\.env\s*\[[^\]]+\]\s*=/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,10 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
describe("shared Vitest setup PostgreSQL inertness", () => {
|
||||
it("guards any optional PostgreSQL setup import behind participation", () => {
|
||||
const source = readFileSync(new URL("../__test-utils__/vitest-setup.ts", import.meta.url), "utf8");
|
||||
expect(source).not.toMatch(/from\s+["'](?:postgres|\.\/pg-test-harness)/);
|
||||
expect(source).not.toMatch(/FUSION_PG_TEST_PREADMISSION_PROBE/);
|
||||
});
|
||||
});
|
||||
@@ -38,6 +38,16 @@ export default mergeConfig(
|
||||
baseConfig,
|
||||
defineConfig({
|
||||
test: {
|
||||
/*
|
||||
FNXC:PgTestPreAdmission 2026-08-17-03:38:
|
||||
FN-9139 requires the dedicated PostgreSQL config to explicitly identify
|
||||
its participating lane. The shared setup stays inert everywhere else,
|
||||
while plain invocations can still opt in through the same environment
|
||||
variable without relying on this config.
|
||||
*/
|
||||
env: {
|
||||
FUSION_PG_TEST_SETUP_PARTICIPANT: "1",
|
||||
},
|
||||
maxWorkers: computeMaxWorkers({ maxCap: PG_MAX_WORKERS }),
|
||||
minWorkers: 1,
|
||||
},
|
||||
|
||||
63
scripts/__tests__/pg-preadmission-campaign.test.mjs
Normal file
63
scripts/__tests__/pg-preadmission-campaign.test.mjs
Normal file
@@ -0,0 +1,63 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
DEFAULT_CAMPAIGN_TIMEOUT_MS,
|
||||
DEFAULT_RUN_TIMEOUT_MS,
|
||||
aggregate,
|
||||
campaignRunEnvironment,
|
||||
classifyLog,
|
||||
countProbeDegradations,
|
||||
decideVerdict,
|
||||
parseArgs,
|
||||
remainingRunTimeoutMs,
|
||||
terminateProcessGroup,
|
||||
} from "../pg-preadmission-campaign.mjs";
|
||||
|
||||
test("parses campaign arguments", () => {
|
||||
assert.deepEqual(parseArgs(["--runs", "5", "--control", "a", "--candidate", "b"]), { runs: 5, scratchDir: undefined, control: "a", candidate: "b" });
|
||||
});
|
||||
|
||||
test("classifies failed files and project identity timeout", () => {
|
||||
assert.deepEqual(classifyLog("Test Files 2 failed | 9 passed\nproject-identity.test.ts hook timed out"), { failingFiles: 2, projectIdentityTimedOut: true });
|
||||
});
|
||||
|
||||
test("counts only degraded preadmission rows", () => {
|
||||
assert.equal(countProbeDegradations('{"event":"pg-preadmission-probe","outcome":"acquired-and-released"}\n{"event":"pg-preadmission-probe","outcome":"connect-failed"}\ninvalid'), 1);
|
||||
});
|
||||
|
||||
test("enables diagnostics and addresses the run process group", () => {
|
||||
assert.deepEqual(campaignRunEnvironment("sample.jsonl"), {
|
||||
FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS: "1",
|
||||
FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_LOG: "sample.jsonl",
|
||||
});
|
||||
const kills = [];
|
||||
terminateProcessGroup({ pid: 42, kill: assert.fail }, "SIGKILL", (...args) => kills.push(args));
|
||||
assert.deepEqual(kills, [[-42, "SIGKILL"]]);
|
||||
});
|
||||
|
||||
test("caps each run by the remaining campaign deadline", () => {
|
||||
assert.equal(remainingRunTimeoutMs(1_000, 1_000), DEFAULT_RUN_TIMEOUT_MS);
|
||||
assert.equal(remainingRunTimeoutMs(1_000, 1_000 + DEFAULT_CAMPAIGN_TIMEOUT_MS - 25), 25);
|
||||
assert.equal(remainingRunTimeoutMs(1_000, 1_000 + DEFAULT_CAMPAIGN_TIMEOUT_MS), 0);
|
||||
});
|
||||
|
||||
const control = Array.from({ length: 5 }, (_, i) => ({ arm: "control", valid: true, wallTimeMs: 100 + i, failingFiles: 2, peakBackends: 10, measurementAvailable: true, projectIdentityTimedOut: false, probeRecords: 0, probeDegradations: null }));
|
||||
const candidate = Array.from({ length: 5 }, (_, i) => ({ arm: "candidate", valid: true, wallTimeMs: 100 + i, failingFiles: 2, peakBackends: 11, measurementAvailable: true, projectIdentityTimedOut: false, probeRecords: 1, probeDegradations: 0 }));
|
||||
test("accepts affordable synthetic evidence", () => assert.equal(decideVerdict([...control, ...candidate]), "boundary-affordable"));
|
||||
test("rejects taxing synthetic evidence", () => assert.equal(decideVerdict([...control, ...candidate.map((run) => ({ ...run, wallTimeMs: 999 }))]), "boundary-taxing"));
|
||||
|
||||
test("retains sampled peak backends and rejects a two-backend increase", () => {
|
||||
const elevatedCandidate = candidate.map((run) => ({ ...run, peakBackends: 12 }));
|
||||
const report = aggregate([...control, ...elevatedCandidate], false);
|
||||
assert.equal(report.candidate.peakBackends, 12);
|
||||
assert.equal(report.verdict, "boundary-taxing");
|
||||
});
|
||||
|
||||
test("rejects incomplete evidence", () => assert.equal(decideVerdict([...control, ...candidate.slice(0, 4)]), "insufficient-data"));
|
||||
|
||||
test("rejects evidence when backend measurement is unavailable", () => {
|
||||
const unavailable = candidate.map((run, index) => index === 0
|
||||
? { ...run, measurementAvailable: false, peakBackends: null }
|
||||
: run);
|
||||
assert.equal(decideVerdict([...control, ...unavailable]), "insufficient-data");
|
||||
});
|
||||
92
scripts/__tests__/pg-setup-boundary-probe.test.mjs
Normal file
92
scripts/__tests__/pg-setup-boundary-probe.test.mjs
Normal file
@@ -0,0 +1,92 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
|
||||
import {
|
||||
classifyVitestOutput,
|
||||
parseProbeArgs,
|
||||
parseProbeEvents,
|
||||
summarizeBoundary,
|
||||
} from "../pg-setup-boundary-probe.mjs";
|
||||
|
||||
/*
|
||||
FNXC:PgSetupBoundaryProbe 2026-08-17-03:25:
|
||||
FN-9139 keeps this report-only survey's unit proof connectionless: fixture strings validate the
|
||||
parser and summary contract without spawning Vitest or making a PostgreSQL admission attempt.
|
||||
*/
|
||||
|
||||
test("parseProbeArgs uses bounded defaults and accepts explicit positive integers", () => {
|
||||
assert.deepEqual(parseProbeArgs([]), { delayMs: 40, timeoutMs: 15, workers: 2 });
|
||||
assert.deepEqual(
|
||||
parseProbeArgs(["--delay-ms", "25", "--timeout-ms", "12", "--workers", "3"]),
|
||||
{ delayMs: 25, timeoutMs: 12, workers: 3 },
|
||||
);
|
||||
assert.deepEqual(parseProbeArgs(["--help"]), { help: true });
|
||||
});
|
||||
|
||||
test("parseProbeArgs rejects unknown, missing, and non-positive values", () => {
|
||||
assert.throws(() => parseProbeArgs(["--unexpected"]), /Unknown argument/);
|
||||
assert.throws(() => parseProbeArgs(["--workers"]), /positive integer/);
|
||||
assert.throws(() => parseProbeArgs(["--delay-ms", "0"]), /positive integer/);
|
||||
assert.throws(() => parseProbeArgs(["--timeout-ms", "1.5"]), /positive integer/);
|
||||
});
|
||||
|
||||
test("parseProbeEvents retains only well-formed prefixed fixture events", () => {
|
||||
const output = [
|
||||
"ordinary Vitest output",
|
||||
'PG_SETUP_BOUNDARY_EVENT:{"type":"boundary","boundary":"global-setup","phase":"start","pid":42}',
|
||||
"PG_SETUP_BOUNDARY_EVENT:not-json",
|
||||
'PG_SETUP_BOUNDARY_EVENT:{"type":"test","boundary":"test","phase":"start","pid":42,"file":"one"}',
|
||||
].join("\n");
|
||||
|
||||
assert.deepEqual(parseProbeEvents(output), [
|
||||
{ type: "boundary", boundary: "global-setup", phase: "start", pid: 42, outputIndex: 1 },
|
||||
{ type: "test", boundary: "test", phase: "start", pid: 42, file: "one", outputIndex: 3 },
|
||||
]);
|
||||
});
|
||||
|
||||
test("classifyVitestOutput separates hook and test budgets from off-budget success", () => {
|
||||
assert.equal(classifyVitestOutput("Error: Hook timed out in 15ms.", 1), "hook-timeout");
|
||||
assert.equal(classifyVitestOutput("Error: Test timed out in 15ms.", 1), "test-timeout");
|
||||
assert.equal(classifyVitestOutput("all tests passed", 0), "off-budget");
|
||||
assert.equal(classifyVitestOutput("unexpected fixture failure", 1), "unknown-failure");
|
||||
});
|
||||
|
||||
test("summarizeBoundary reports per-file execution and ordering before the first test", () => {
|
||||
const normalRun = {
|
||||
exitCode: 0,
|
||||
output: [
|
||||
'PG_SETUP_BOUNDARY_EVENT:{"type":"boundary","boundary":"per-file-before-all","phase":"start","pid":11,"file":"one"}',
|
||||
'PG_SETUP_BOUNDARY_EVENT:{"type":"boundary","boundary":"per-file-before-all","phase":"start","pid":12,"file":"two"}',
|
||||
'PG_SETUP_BOUNDARY_EVENT:{"type":"test","boundary":"test","phase":"start","pid":11,"file":"one"}',
|
||||
].join("\n"),
|
||||
};
|
||||
const stressedRun = { exitCode: 1, output: "Error: Hook timed out in 15ms." };
|
||||
|
||||
assert.deepEqual(summarizeBoundary("per-file-before-all", normalRun, stressedRun), {
|
||||
boundary: "per-file-before-all",
|
||||
granularity: "per file",
|
||||
executions: 2,
|
||||
workersObserved: 2,
|
||||
filesObserved: 2,
|
||||
timeoutCharge: "hook-timeout",
|
||||
orderingRelativeToFirstTest: "before-first-test",
|
||||
});
|
||||
});
|
||||
|
||||
test("summarizeBoundary uses global invocation semantics and reports absent ordering honestly", () => {
|
||||
const normalRun = {
|
||||
exitCode: 0,
|
||||
output: 'PG_SETUP_BOUNDARY_EVENT:{"type":"boundary","boundary":"global-setup","phase":"start","pid":99}',
|
||||
};
|
||||
const stressedRun = { exitCode: 0, output: "passed" };
|
||||
|
||||
assert.deepEqual(summarizeBoundary("global-setup", normalRun, stressedRun), {
|
||||
boundary: "global-setup",
|
||||
granularity: "per invocation",
|
||||
executions: 1,
|
||||
workersObserved: 1,
|
||||
filesObserved: 0,
|
||||
timeoutCharge: "off-budget",
|
||||
orderingRelativeToFirstTest: "not-observed",
|
||||
});
|
||||
});
|
||||
194
scripts/pg-preadmission-campaign.mjs
Normal file
194
scripts/pg-preadmission-campaign.mjs
Normal file
@@ -0,0 +1,194 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { spawn } from "node:child_process";
|
||||
import { join, resolve } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
/*
|
||||
FNXC:PgTestPreAdmission 2026-08-17-03:20:
|
||||
FN-9139 requires pre-admission affordability evidence to be collected by one
|
||||
bounded shell, so a missing operator terminal cannot turn measurement into an
|
||||
unbounded handoff. The driver alternates arms and treats incomplete or invalid
|
||||
samples as insufficient data instead of retrying toward a preferred outcome.
|
||||
*/
|
||||
export const MINIMUM_RUNS_PER_ARM = 5;
|
||||
export const DEFAULT_RUN_TIMEOUT_MS = 300_000;
|
||||
export const DEFAULT_CAMPAIGN_TIMEOUT_MS = 3_600_000;
|
||||
|
||||
export function parseArgs(args) {
|
||||
const result = { runs: MINIMUM_RUNS_PER_ARM, scratchDir: undefined, control: undefined, candidate: undefined };
|
||||
for (let i = 0; i < args.length; i += 1) {
|
||||
const value = args[i];
|
||||
if (value === "--runs") result.runs = Number(args[++i]);
|
||||
else if (value === "--scratch-dir") result.scratchDir = args[++i];
|
||||
else if (value === "--control") result.control = args[++i];
|
||||
else if (value === "--candidate") result.candidate = args[++i];
|
||||
else throw new Error(`Unknown argument: ${value}`);
|
||||
}
|
||||
if (!Number.isInteger(result.runs) || result.runs < 1) throw new Error("--runs must be a positive integer");
|
||||
return result;
|
||||
}
|
||||
|
||||
export function stripAnsi(text) {
|
||||
return text.replace(new RegExp(`${String.fromCharCode(27)}\\[[0-?]*[ -/]*[@-~]`, "g"), "");
|
||||
}
|
||||
|
||||
export function classifyLog(log) {
|
||||
const clean = stripAnsi(log);
|
||||
const failed = [...clean.matchAll(/^\s*Test Files\s+.*?(\d+)\s+failed\b/gim)].at(-1);
|
||||
return {
|
||||
failingFiles: failed ? Number(failed[1]) : 0,
|
||||
projectIdentityTimedOut: /project-identity\.test\.ts[\s\S]{0,500}?timed out/i.test(clean),
|
||||
};
|
||||
}
|
||||
|
||||
export function countProbeDegradations(jsonl) {
|
||||
return jsonl.split(/\r?\n/).filter(Boolean).reduce((count, line) => {
|
||||
try {
|
||||
const row = JSON.parse(line);
|
||||
return count + (row.event === "pg-preadmission-probe" && row.outcome !== "acquired-and-released" ? 1 : 0);
|
||||
} catch { return count; }
|
||||
}, 0);
|
||||
}
|
||||
|
||||
export function median(values) {
|
||||
const sorted = values.filter(Number.isFinite).toSorted((a, b) => a - b);
|
||||
return sorted.length === 0 ? null : sorted[Math.floor(sorted.length / 2)];
|
||||
}
|
||||
|
||||
export function decideVerdict(runs, { incomplete = false } = {}) {
|
||||
const control = runs.filter((run) => run.arm === "control" && run.valid);
|
||||
const candidate = runs.filter((run) => run.arm === "candidate" && run.valid);
|
||||
const measurementsAvailable = [...control, ...candidate].every((run) => run.measurementAvailable === true && Number.isFinite(run.peakBackends));
|
||||
if (incomplete || control.length < MINIMUM_RUNS_PER_ARM || candidate.length < MINIMUM_RUNS_PER_ARM || !measurementsAvailable) return "insufficient-data";
|
||||
if (candidate.some((run) => run.probeRecords === 0 || run.probeDegradations !== 0)) return "boundary-taxing";
|
||||
const controlMaxWall = Math.max(...control.map((run) => run.wallTimeMs));
|
||||
const controlMaxFailures = Math.max(...control.map((run) => run.failingFiles));
|
||||
const controlPeak = Math.max(...control.map((run) => run.peakBackends));
|
||||
const candidateMedian = median(candidate.map((run) => run.wallTimeMs));
|
||||
const candidateMaxFailures = Math.max(...candidate.map((run) => run.failingFiles));
|
||||
const candidatePeak = Math.max(...candidate.map((run) => run.peakBackends));
|
||||
const identityWorsened = candidate.some((run) => run.projectIdentityTimedOut) && !control.some((run) => run.projectIdentityTimedOut);
|
||||
return candidateMedian <= controlMaxWall && candidateMaxFailures <= controlMaxFailures && candidatePeak <= controlPeak + 1 && !identityWorsened
|
||||
? "boundary-affordable" : "boundary-taxing";
|
||||
}
|
||||
|
||||
export function aggregate(runs, incomplete) {
|
||||
const summarize = (arm) => {
|
||||
const selected = runs.filter((run) => run.arm === arm);
|
||||
return { count: selected.length, medianWallTimeMs: median(selected.map((run) => run.wallTimeMs)), maxWallTimeMs: Math.max(...selected.map((run) => run.wallTimeMs), 0), maxFailingFiles: Math.max(...selected.map((run) => run.failingFiles), 0), peakBackends: Math.max(...selected.map((run) => run.peakBackends), 0) };
|
||||
};
|
||||
return { runs, control: summarize("control"), candidate: summarize("candidate"), incomplete, verdict: decideVerdict(runs, { incomplete }) };
|
||||
}
|
||||
|
||||
export function terminateProcessGroup(child, signal, kill = process.kill) {
|
||||
if (!child.pid) return;
|
||||
try {
|
||||
kill(-child.pid, signal);
|
||||
} catch {
|
||||
try { child.kill(signal); } catch { /* The process already exited. */ }
|
||||
}
|
||||
}
|
||||
|
||||
export function campaignRunEnvironment(diagnostics) {
|
||||
return {
|
||||
FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS: "1",
|
||||
FUSION_PG_TEST_TEARDOWN_DIAGNOSTICS_LOG: diagnostics,
|
||||
};
|
||||
}
|
||||
|
||||
export function remainingRunTimeoutMs(startedAt, now = Date.now()) {
|
||||
return Math.min(DEFAULT_RUN_TIMEOUT_MS, DEFAULT_CAMPAIGN_TIMEOUT_MS - (now - startedAt));
|
||||
}
|
||||
|
||||
export async function runCommand(command, env, timeoutMs) {
|
||||
return await new Promise((resolveRun) => {
|
||||
const startedAt = Date.now();
|
||||
// process-supervisor-allowlist: isolation is required so the bounded campaign can kill the shell command's entire process group
|
||||
const child = spawn(command, { shell: true, detached: true, env: { ...process.env, ...env } });
|
||||
// FNXC:PgTestPreAdmission 2026-08-17-04:05:
|
||||
// The timeout owns the shell's whole process group. Killing only the shell
|
||||
// can orphan Vitest or its workers and violate both campaign deadlines.
|
||||
const samplerScript = `while kill -0 ${child.pid} 2>/dev/null; do if value=$(psql --no-password -X -Atqc 'select count(*) from pg_stat_activity;' postgres 2>/dev/null); then printf 'SAMPLE:%s\\n' "$value"; else echo SAMPLE_ERROR; exit 1; fi; sleep 1; done`;
|
||||
// process-supervisor-allowlist: the sampler shares the campaign's bounded process-group cleanup contract
|
||||
const sampler = spawn(samplerScript, { shell: true, detached: true, stdio: ["ignore", "pipe", "ignore"] });
|
||||
let output = "";
|
||||
let sampled = "";
|
||||
let childResult;
|
||||
let samplerClosed = false;
|
||||
let samplerErrored = false;
|
||||
let timedOut = false;
|
||||
const finish = () => {
|
||||
if (!childResult || !samplerClosed) return;
|
||||
const samples = [...sampled.matchAll(/^SAMPLE:(\d+)$/gm)].map((match) => Number(match[1]));
|
||||
const measurementAvailable = !samplerErrored && !sampled.includes("SAMPLE_ERROR") && samples.length > 0;
|
||||
resolveRun({
|
||||
output,
|
||||
...childResult,
|
||||
wallTimeMs: Date.now() - startedAt,
|
||||
peakBackends: measurementAvailable ? Math.max(...samples) : null,
|
||||
measurementAvailable,
|
||||
timedOut,
|
||||
});
|
||||
};
|
||||
child.stdout.on("data", (chunk) => { output += chunk; });
|
||||
child.stderr.on("data", (chunk) => { output += chunk; });
|
||||
sampler.stdout.on("data", (chunk) => { sampled += chunk; });
|
||||
sampler.once("error", () => { samplerErrored = true; samplerClosed = true; finish(); });
|
||||
sampler.once("close", () => { samplerClosed = true; finish(); });
|
||||
const timeout = setTimeout(() => {
|
||||
timedOut = true;
|
||||
terminateProcessGroup(child, "SIGTERM");
|
||||
terminateProcessGroup(child, "SIGKILL");
|
||||
}, timeoutMs);
|
||||
child.once("close", (code, signal) => {
|
||||
clearTimeout(timeout);
|
||||
childResult = { code, signal };
|
||||
terminateProcessGroup(sampler, "SIGTERM");
|
||||
terminateProcessGroup(sampler, "SIGKILL");
|
||||
finish();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const args = parseArgs(process.argv.slice(2));
|
||||
if (!args.control || !args.candidate) throw new Error("Supply --control and --candidate commands");
|
||||
const scratch = resolve(args.scratchDir ?? join(tmpdir(), `fusion-pg-preadmission-${process.pid}`));
|
||||
mkdirSync(scratch, { recursive: true });
|
||||
const startedAt = Date.now();
|
||||
const results = [];
|
||||
let incomplete = false;
|
||||
for (let index = 0; index < args.runs * 2; index += 1) {
|
||||
const runTimeoutMs = remainingRunTimeoutMs(startedAt);
|
||||
if (runTimeoutMs <= 0) { incomplete = true; break; }
|
||||
const arm = index % 2 === 0 ? "control" : "candidate";
|
||||
const diagnostics = join(scratch, `${String(index + 1).padStart(2, "0")}-${arm}.jsonl`);
|
||||
// FNXC:PgTestPreAdmission 2026-08-17-04:05:
|
||||
// Diagnostics must be explicitly enabled, and an arm may consume only the
|
||||
// smaller of its five-minute allowance and the campaign's remaining time.
|
||||
const result = await runCommand(
|
||||
arm === "control" ? args.control : args.candidate,
|
||||
campaignRunEnvironment(diagnostics),
|
||||
runTimeoutMs,
|
||||
);
|
||||
const logPath = join(scratch, `${String(index + 1).padStart(2, "0")}-${arm}.log`);
|
||||
writeFileSync(logPath, result.output);
|
||||
const parsed = classifyLog(result.output);
|
||||
const jsonl = (() => { try { return readFileSync(diagnostics, "utf8"); } catch { return ""; } })();
|
||||
const probeRecords = [...jsonl.matchAll(/"event":"pg-preadmission-probe"/g)].length;
|
||||
// FNXC:PgTestPreAdmission 2026-08-17-03:38:
|
||||
// FN-9139's affordability verdict must retain the sampler's observed peak.
|
||||
// Replacing it would falsely certify a candidate whose extra backend exceeds
|
||||
// the one-connection observation-only footprint.
|
||||
results.push({ arm, ...result, ...parsed, probeRecords, probeDegradations: arm === "control" ? null : countProbeDegradations(jsonl), valid: !result.timedOut && result.signal === null && result.measurementAvailable && (arm === "control" || probeRecords > 0) });
|
||||
if (result.timedOut || Date.now() - startedAt >= DEFAULT_CAMPAIGN_TIMEOUT_MS) { incomplete = true; break; }
|
||||
}
|
||||
const report = aggregate(results, incomplete);
|
||||
writeFileSync(join(scratch, "results.json"), `${JSON.stringify(report, null, 2)}\n`);
|
||||
console.log(JSON.stringify({ ...report, scratchDir: scratch }, null, 2));
|
||||
}
|
||||
|
||||
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) main().catch((error) => { console.error(error); process.exitCode = 1; });
|
||||
182
scripts/pg-setup-boundary-probe.mjs
Normal file
182
scripts/pg-setup-boundary-probe.mjs
Normal file
@@ -0,0 +1,182 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { createRequire } from "node:module";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
/*
|
||||
FNXC:PgSetupBoundaryProbe 2026-08-17-03:25:
|
||||
FN-9139 must establish whether Vitest lifecycle setup can host a future PostgreSQL admission
|
||||
without changing the PostgreSQL harness. This isolated fixture observes only lifecycle timing and
|
||||
never loads a repository config, a database client, or a project test suite.
|
||||
*/
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const BOUNDARIES = ["global-setup", "setup-top-level-await", "setup-before-all", "per-file-before-all"];
|
||||
const DEFAULTS = Object.freeze({ delayMs: 40, timeoutMs: 15, workers: 2 });
|
||||
|
||||
export function parseProbeArgs(argv) {
|
||||
const options = { ...DEFAULTS };
|
||||
const names = new Map([
|
||||
["--delay-ms", "delayMs"],
|
||||
["--timeout-ms", "timeoutMs"],
|
||||
["--workers", "workers"],
|
||||
]);
|
||||
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
const argument = argv[index];
|
||||
if (argument === "--help") return { help: true };
|
||||
const name = names.get(argument);
|
||||
if (!name) throw new Error(`Unknown argument: ${argument}`);
|
||||
const value = Number(argv[++index]);
|
||||
if (!Number.isSafeInteger(value) || value <= 0) {
|
||||
throw new Error(`${argument} must be a positive integer`);
|
||||
}
|
||||
options[name] = value;
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
/** Extract deliberately prefixed fixture events without coupling classification to Vitest's reporter. */
|
||||
export function parseProbeEvents(output) {
|
||||
return output.split(/\r?\n/).flatMap((line, index) => {
|
||||
const marker = line.indexOf("PG_SETUP_BOUNDARY_EVENT:");
|
||||
if (marker < 0) return [];
|
||||
try {
|
||||
const event = JSON.parse(line.slice(marker + "PG_SETUP_BOUNDARY_EVENT:".length));
|
||||
return [{ ...event, outputIndex: index }];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** Classify timeout ownership from stable Vitest diagnostic phrases, preserving unknown output. */
|
||||
export function classifyVitestOutput(output, exitCode = 0) {
|
||||
if (/Hook timed out in \d+ms\./i.test(output)) return "hook-timeout";
|
||||
if (/Test timed out in \d+ms\./i.test(output)) return "test-timeout";
|
||||
if (exitCode === 0) return "off-budget";
|
||||
return "unknown-failure";
|
||||
}
|
||||
|
||||
export function summarizeBoundary(boundary, normalRun, stressedRun) {
|
||||
const events = parseProbeEvents(normalRun.output);
|
||||
const starts = events.filter((event) => event.boundary === boundary && event.phase === "start");
|
||||
const firstTest = events.find((event) => event.type === "test" && event.phase === "start");
|
||||
const lastBoundary = starts.at(-1);
|
||||
const files = new Set(starts.map((event) => event.file).filter(Boolean));
|
||||
const pids = new Set(starts.map((event) => event.pid).filter(Boolean));
|
||||
const granularity = boundary === "global-setup"
|
||||
? "per invocation"
|
||||
: files.size > 1
|
||||
? "per file"
|
||||
: pids.size > 1
|
||||
? "per worker"
|
||||
: "per invocation";
|
||||
|
||||
return {
|
||||
boundary,
|
||||
granularity,
|
||||
executions: starts.length,
|
||||
workersObserved: pids.size,
|
||||
filesObserved: files.size,
|
||||
timeoutCharge: classifyVitestOutput(stressedRun.output, stressedRun.exitCode),
|
||||
orderingRelativeToFirstTest: !firstTest || !lastBoundary
|
||||
? "not-observed"
|
||||
: lastBoundary.outputIndex < firstTest.outputIndex
|
||||
? "before-first-test"
|
||||
: "after-first-test",
|
||||
};
|
||||
}
|
||||
|
||||
function eventSource(type, boundary, phase, file = "") {
|
||||
return `console.log("PG_SETUP_BOUNDARY_EVENT:" + JSON.stringify({type:${JSON.stringify(type)},boundary:${JSON.stringify(boundary)},phase:${JSON.stringify(phase)},pid:process.pid,file:${JSON.stringify(file)}}));`;
|
||||
}
|
||||
|
||||
function fixtureFiles(boundary, delayMs, timeoutMs, workers, vitestApiUrl) {
|
||||
const delay = `await new Promise((resolve) => setTimeout(resolve, ${delayMs}));`;
|
||||
// The fixture is outside the repository, so its Vitest API import must be absolute.
|
||||
const config = `export default { test: { include: ["tests/**/*.test.mjs"], pool: "forks", maxWorkers: ${workers}, minWorkers: ${workers}, testTimeout: ${timeoutMs}, hookTimeout: ${timeoutMs}, setupFiles: ["./setup.mjs"]${boundary === "global-setup" ? ', globalSetup: ["./global-setup.mjs"]' : ""} } };\n`;
|
||||
const setup = boundary === "setup-top-level-await"
|
||||
? `${eventSource("boundary", boundary, "start")}${delay}${eventSource("boundary", boundary, "end")}`
|
||||
: boundary === "setup-before-all"
|
||||
? `import { beforeAll } from ${JSON.stringify(vitestApiUrl)};\nbeforeAll(async () => { ${eventSource("boundary", boundary, "start")}${delay}${eventSource("boundary", boundary, "end")} });`
|
||||
: "";
|
||||
const global = boundary === "global-setup"
|
||||
? `export default async function setup() { ${eventSource("boundary", boundary, "start")}${delay}${eventSource("boundary", boundary, "end")} }`
|
||||
: "export default async function setup() {}";
|
||||
const test = (file) => `import { beforeAll, test } from ${JSON.stringify(vitestApiUrl)};\n${boundary === "per-file-before-all" ? `beforeAll(async () => { ${eventSource("boundary", boundary, "start", file)}${delay}${eventSource("boundary", boundary, "end", file)} });` : ""}\ntest(${JSON.stringify(file)}, () => { ${eventSource("test", "test", "start", file)} });\n`;
|
||||
return {
|
||||
"vitest.config.mjs": config,
|
||||
"setup.mjs": setup,
|
||||
"global-setup.mjs": global,
|
||||
"tests/one.test.mjs": test("one"),
|
||||
"tests/two.test.mjs": test("two"),
|
||||
};
|
||||
}
|
||||
|
||||
async function writeFixture(directory, files) {
|
||||
await Promise.all(Object.entries(files).map(async ([file, content]) => {
|
||||
const path = join(directory, file);
|
||||
await mkdir(dirname(path), { recursive: true });
|
||||
await writeFile(path, content, "utf8");
|
||||
}));
|
||||
}
|
||||
|
||||
function runVitest(directory) {
|
||||
const vitest = join(dirname(require.resolve("vitest", { paths: [resolve(process.cwd(), "packages/core")] })), "vitest.mjs");
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(process.execPath, [vitest, "run", "--config", "vitest.config.mjs", "--reporter=verbose"], {
|
||||
cwd: directory,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
let output = "";
|
||||
child.stdout.on("data", (chunk) => { output += chunk; });
|
||||
child.stderr.on("data", (chunk) => { output += chunk; });
|
||||
child.on("error", reject);
|
||||
child.on("close", (exitCode) => resolve({ output, exitCode: exitCode ?? 1 }));
|
||||
});
|
||||
}
|
||||
|
||||
async function runBoundary(boundary, options, stressed) {
|
||||
const directory = await mkdtemp(join(tmpdir(), "fusion-pg-setup-boundary-"));
|
||||
try {
|
||||
const delayMs = stressed ? Math.max(options.delayMs, options.timeoutMs + 20) : Math.min(options.delayMs, Math.max(1, options.timeoutMs - 1));
|
||||
const vitestApiUrl = pathToFileURL(require.resolve("vitest", { paths: [resolve(process.cwd(), "packages/core")] })).href;
|
||||
await writeFixture(directory, fixtureFiles(boundary, delayMs, options.timeoutMs, options.workers, vitestApiUrl));
|
||||
return await runVitest(directory);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export async function runProbe(options) {
|
||||
const summaries = [];
|
||||
for (const boundary of BOUNDARIES) {
|
||||
const normalRun = await runBoundary(boundary, options, false);
|
||||
const stressedRun = await runBoundary(boundary, options, true);
|
||||
summaries.push(summarizeBoundary(boundary, normalRun, stressedRun));
|
||||
}
|
||||
return summaries;
|
||||
}
|
||||
|
||||
function printHelp() {
|
||||
console.log("Usage: node scripts/pg-setup-boundary-probe.mjs [--delay-ms N] [--timeout-ms N] [--workers N]");
|
||||
}
|
||||
|
||||
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
|
||||
try {
|
||||
const options = parseProbeArgs(process.argv.slice(2));
|
||||
if (options.help) {
|
||||
printHelp();
|
||||
} else {
|
||||
console.table(await runProbe(options));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`[pg-setup-boundary-probe] ${error.message}`);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user