FN-9158: recover Windows antivirus-blocked PostgreSQL payloads
Recover quarantined or truncated embedded PostgreSQL runtime files and explain Windows Defender remediation. - Validate materialized payload inventory with v3 markers and repair mismatches atomically. - Surface actionable error 4551 diagnostics through startup failures. - Add regression coverage, operator documentation, and a patch changeset. Files changed: ...-9158-windows-av-blocked-embedded-pg-payload.md | 7 + ...ndows-antivirus-blocks-embedded-postgres-dll.md | 27 +++ docs/storage.md | 4 + .../__tests__/postgres/embedded-lifecycle.test.ts | 139 +++++++++++++- .../src/__tests__/postgres/startup-factory.test.ts | 19 ++ packages/core/src/postgres/embedded-lifecycle.ts | 208 ++++++++++++++++++++- packages/core/src/postgres/startup-factory.ts | 30 +-- 7 files changed, 414 insertions(+), 20 deletions(-) Fusion-Task-Id: FN-9158 Fusion-Task-Lineage: ec2c97d7-0797-49c1-b875-6b147ff26917 Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
@@ -0,0 +1,7 @@
|
|||||||
|
---
|
||||||
|
"@runfusion/fusion": patch
|
||||||
|
---
|
||||||
|
|
||||||
|
summary: Recover automatically when Windows antivirus blocks a bundled PostgreSQL library.
|
||||||
|
category: fix
|
||||||
|
dev: Marker v3 verifies cached payload inventory, failed verification leaves no marker, and reports EmbeddedPostgresPayloadBlockedError.
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
---
|
||||||
|
category: database-issues
|
||||||
|
module: embedded-postgres
|
||||||
|
tags: [windows, antivirus, postgresql, embedded-postgres]
|
||||||
|
problem_type: installation
|
||||||
|
applies_when: Windows startup reports `could not load library` with `unknown error 4551` for an embedded PostgreSQL DLL.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Windows antivirus blocks an embedded PostgreSQL DLL
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
Fusion fails during embedded PostgreSQL initialization with a message such as `could not load library .../lib/dict_snowball.dll: unknown error 4551`.
|
||||||
|
|
||||||
|
## Cause
|
||||||
|
|
||||||
|
Windows error 4551 is `ERROR_VIRUS_DELETED`: Windows Defender or another antivirus product quarantined a DLL from Fusion's host-local embedded PostgreSQL runtime payload. See [issue #3489](https://github.com/Runfusion/Fusion/issues/3489).
|
||||||
|
|
||||||
|
Earlier versions trusted a marker that identified the source payload but did not verify the copied destination. A quarantined destination DLL could therefore remain cached across every restart.
|
||||||
|
|
||||||
|
## Remedy
|
||||||
|
|
||||||
|
1. In **Windows Security**, open **Virus & threat protection** → **Manage settings** → **Exclusions** and add `%USERPROFILE%\.fusion\embedded-postgres`.
|
||||||
|
2. Restore the quarantined DLL from **Protection history**.
|
||||||
|
3. Restart Fusion.
|
||||||
|
|
||||||
|
Fusion verifies the runtime-bin payload on startup. Once the exclusion permits the copy, Fusion automatically re-materializes missing or truncated files and clears the recovery diagnosis.
|
||||||
@@ -847,6 +847,10 @@ Project-scoped structured recall records for durable decisions, preferences, and
|
|||||||
|
|
||||||
- Knowledge-graph artifact: `<rootDir>/.fusion-knowledge/graph/` (`nodes.json`, `edges.json`, and `manifest.json`). This is deliberately outside ignored `.fusion` and may be committed at the operator's discretion.
|
- Knowledge-graph artifact: `<rootDir>/.fusion-knowledge/graph/` (`nodes.json`, `edges.json`, and `manifest.json`). This is deliberately outside ignored `.fusion` and may be committed at the operator's discretion.
|
||||||
|
|
||||||
|
### Embedded PostgreSQL on Windows — antivirus and the runtime-bin payload
|
||||||
|
|
||||||
|
If Windows startup reports `unknown error 4551` while loading an embedded PostgreSQL DLL such as `dict_snowball.dll`, antivirus quarantined part of `%USERPROFILE%\\.fusion\\embedded-postgres`. Add that directory as a Windows Security exclusion, restore the file from Protection history, and restart Fusion. Fusion verifies and automatically re-copies the runtime payload; see [Windows antivirus blocks an embedded PostgreSQL DLL](solutions/database-issues/windows-antivirus-blocks-embedded-postgres-dll.md).
|
||||||
|
|
||||||
### Bounded task-intake lookups
|
### Bounded task-intake lookups
|
||||||
|
|
||||||
Recommendation proposal claims use the indexed `findTaskByProposalClaimId` read (`uqTasksProjectProposalClaimId`), and same-agent intake reads only matching source lineage (`idxTasksProjectSourceAgentId` and `idxTasksSourceParentTaskId`). Do not replace either read with a `listTasks()` scan. Workflow terminal flags for intake duplicate checks are derived from workflow definitions, not board rows. Guarded-intake near-duplicate checks must remain bounded to their candidates (the fallback is `limit: 50`) and must not hydrate the full board.
|
Recommendation proposal claims use the indexed `findTaskByProposalClaimId` read (`uqTasksProjectProposalClaimId`), and same-agent intake reads only matching source lineage (`idxTasksProjectSourceAgentId` and `idxTasksSourceParentTaskId`). Do not replace either read with a `listTasks()` scan. Workflow terminal flags for intake duplicate checks are derived from workflow definitions, not board rows. Guarded-intake near-duplicate checks must remain bounded to their candidates (the fallback is `limit: 50`) and must not hydrate the full board.
|
||||||
|
|||||||
@@ -56,6 +56,14 @@ import {
|
|||||||
materializeEmbeddedPostgresRuntimeBinaries,
|
materializeEmbeddedPostgresRuntimeBinaries,
|
||||||
installElectronAsarNativePathPatch,
|
installElectronAsarNativePathPatch,
|
||||||
uninstallElectronAsarNativePathPatchForTests,
|
uninstallElectronAsarNativePathPatchForTests,
|
||||||
|
isWindowsBlockedNativeLibraryError,
|
||||||
|
describeWindowsBlockedNativeLibraryError,
|
||||||
|
EmbeddedPostgresPayloadBlockedError,
|
||||||
|
recordEmbeddedPayloadIntegrityFailure,
|
||||||
|
getEmbeddedPayloadIntegrityFailure,
|
||||||
|
clearEmbeddedPayloadIntegrityFailure,
|
||||||
|
decorateWindowsBlockedNativeLibraryError,
|
||||||
|
embeddedPostgresRuntimeBinRoot,
|
||||||
type EmbeddedLifecycleOptions,
|
type EmbeddedLifecycleOptions,
|
||||||
} from "../../postgres/embedded-lifecycle.js";
|
} from "../../postgres/embedded-lifecycle.js";
|
||||||
|
|
||||||
@@ -75,6 +83,7 @@ afterEach(async () => {
|
|||||||
__setWindowsElevatedAdminForTests(null);
|
__setWindowsElevatedAdminForTests(null);
|
||||||
__setWindowsEmbeddedPostgresNativeRootForTests(null);
|
__setWindowsEmbeddedPostgresNativeRootForTests(null);
|
||||||
__setWindowsLauncherForTests(null);
|
__setWindowsLauncherForTests(null);
|
||||||
|
clearEmbeddedPayloadIntegrityFailure();
|
||||||
vi.useRealTimers();
|
vi.useRealTimers();
|
||||||
while (tracked.length > 0) {
|
while (tracked.length > 0) {
|
||||||
const { lifecycle, dataDir } = tracked.pop()!;
|
const { lifecycle, dataDir } = tracked.pop()!;
|
||||||
@@ -124,6 +133,27 @@ describe("embedded-lifecycle: isDataDirInitialized (PG_VERSION marker)", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("embedded-lifecycle: Windows blocked native library classifier", () => {
|
||||||
|
const issueOutput = `ERROR OUTPUT: 2026-08-19 09:20:48.546 CEST [23152] FATAL: could not load library "C:/Users/ppp/.fusion/embedded-postgres/runtime-bin/win32-x64/lib/dict_snowball.dll": unknown error 4551`;
|
||||||
|
|
||||||
|
it("classifies the Defender ERROR_VIRUS_DELETED failure from issue #3489", () => {
|
||||||
|
expect(isWindowsBlockedNativeLibraryError(issueOutput)).toBe(true);
|
||||||
|
expect(describeWindowsBlockedNativeLibraryError(issueOutput)).toContain("dict_snowball.dll");
|
||||||
|
expect(describeWindowsBlockedNativeLibraryError(issueOutput)).toContain(
|
||||||
|
"%USERPROFILE%\\.fusion\\embedded-postgres",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not misclassify unrelated library or encoding errors", () => {
|
||||||
|
expect(isWindowsBlockedNativeLibraryError(
|
||||||
|
"could not load library: The specified module could not be found",
|
||||||
|
)).toBe(false);
|
||||||
|
expect(isWindowsBlockedNativeLibraryError(
|
||||||
|
"invalid byte sequence for encoding UTF8",
|
||||||
|
)).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe("embedded-lifecycle: Windows elevation probe (no process)", () => {
|
describe("embedded-lifecycle: Windows elevation probe (no process)", () => {
|
||||||
it("isWindowsElevatedAdmin is false on non-Windows platforms", () => {
|
it("isWindowsElevatedAdmin is false on non-Windows platforms", () => {
|
||||||
// FNXC:WindowsDesktopPackaging 2026-07-15-04:55:
|
// FNXC:WindowsDesktopPackaging 2026-07-15-04:55:
|
||||||
@@ -303,6 +333,33 @@ describe("embedded-lifecycle: Electron asar unpacked path rewrite", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("embedded-lifecycle: payload integrity diagnosis lifecycle", () => {
|
||||||
|
const verification = { acceptable: false, mismatches: ["lib/dict_snowball.dll"], mismatchCount: 1 };
|
||||||
|
|
||||||
|
it("is root-scoped, latest-wins, and resettable", () => {
|
||||||
|
const first = new EmbeddedPostgresPayloadBlockedError("/native-a", "/runtime-a", verification);
|
||||||
|
const second = new EmbeddedPostgresPayloadBlockedError("/native-b", "/runtime-b", verification);
|
||||||
|
recordEmbeddedPayloadIntegrityFailure(first);
|
||||||
|
recordEmbeddedPayloadIntegrityFailure(second);
|
||||||
|
expect(getEmbeddedPayloadIntegrityFailure("/runtime-a")).toBeNull();
|
||||||
|
expect(getEmbeddedPayloadIntegrityFailure("/runtime-b")).toBe(second);
|
||||||
|
clearEmbeddedPayloadIntegrityFailure();
|
||||||
|
expect(getEmbeddedPayloadIntegrityFailure()).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("decorates only matching-root start failures and preserves unrelated failures", () => {
|
||||||
|
const root = embeddedPostgresRuntimeBinRoot();
|
||||||
|
const failure = new EmbeddedPostgresPayloadBlockedError("/native", root, verification);
|
||||||
|
recordEmbeddedPayloadIntegrityFailure(failure);
|
||||||
|
expect(decorateWindowsBlockedNativeLibraryError(new Error("initdb failed"), root).message).toContain(
|
||||||
|
"Windows antivirus blocked a bundled PostgreSQL library",
|
||||||
|
);
|
||||||
|
expect(decorateWindowsBlockedNativeLibraryError(new Error("initdb failed"), "/other-root").message).toBe(
|
||||||
|
"initdb failed",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe("embedded-lifecycle: materialize runtime binaries (update-safe marker)", () => {
|
describe("embedded-lifecycle: materialize runtime binaries (update-safe marker)", () => {
|
||||||
/*
|
/*
|
||||||
* FNXC:DesktopEmbeddedPostgres 2026-07-15-02:55:
|
* FNXC:DesktopEmbeddedPostgres 2026-07-15-02:55:
|
||||||
@@ -327,6 +384,7 @@ describe("embedded-lifecycle: materialize runtime binaries (update-safe marker)"
|
|||||||
writeFileSync(join(root, "bin", initdbBin), "initdb-stub");
|
writeFileSync(join(root, "bin", initdbBin), "initdb-stub");
|
||||||
writeFileSync(join(root, "bin", pgCtlBin), "pg_ctl-stub");
|
writeFileSync(join(root, "bin", pgCtlBin), "pg_ctl-stub");
|
||||||
writeFileSync(join(root, "lib", "postgresql", "plpgsql.so"), "ext-v1");
|
writeFileSync(join(root, "lib", "postgresql", "plpgsql.so"), "ext-v1");
|
||||||
|
writeFileSync(join(root, "lib", "dict_snowball.dll"), "snowball-v1");
|
||||||
writeFileSync(join(root, "share", "postgresql", "postgres.bki"), "share-v1");
|
writeFileSync(join(root, "share", "postgresql", "postgres.bki"), "share-v1");
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -400,7 +458,7 @@ describe("embedded-lifecycle: materialize runtime binaries (update-safe marker)"
|
|||||||
seedNativeRoot(nativeRoot, "postgres-body");
|
seedNativeRoot(nativeRoot, "postgres-body");
|
||||||
const marker = buildEmbeddedPostgresMaterializationMarker(nativeRoot);
|
const marker = buildEmbeddedPostgresMaterializationMarker(nativeRoot);
|
||||||
const fingerprint = fingerprintEmbeddedPostgresNativeRoot(nativeRoot);
|
const fingerprint = fingerprintEmbeddedPostgresNativeRoot(nativeRoot);
|
||||||
expect(marker.startsWith("v2\n")).toBe(true);
|
expect(marker.startsWith("v3\n")).toBe(true);
|
||||||
expect(marker).toContain(nativeRoot);
|
expect(marker).toContain(nativeRoot);
|
||||||
expect(marker).toContain(fingerprint);
|
expect(marker).toContain(fingerprint);
|
||||||
// Path alone must not equal the full marker (legacy path-only markers rematerialize).
|
// Path alone must not equal the full marker (legacy path-only markers rematerialize).
|
||||||
@@ -437,6 +495,62 @@ describe("embedded-lifecycle: materialize runtime binaries (update-safe marker)"
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("clears a recorded integrity failure after a verified repair", () => {
|
||||||
|
const nativeRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-repair-src-"));
|
||||||
|
const destRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-repair-dst-"));
|
||||||
|
try {
|
||||||
|
seedNativeRoot(nativeRoot, "postgres-stable");
|
||||||
|
recordEmbeddedPayloadIntegrityFailure(
|
||||||
|
new EmbeddedPostgresPayloadBlockedError(nativeRoot, destRoot, {
|
||||||
|
acceptable: false,
|
||||||
|
mismatches: ["lib/dict_snowball.dll"],
|
||||||
|
mismatchCount: 1,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
materializeEmbeddedPostgresRuntimeBinaries(nativeRoot, { destRoot });
|
||||||
|
expect(existsSync(join(destRoot, ".materialized-from"))).toBe(true);
|
||||||
|
expect(getEmbeddedPayloadIntegrityFailure(destRoot)).toBeNull();
|
||||||
|
expect(decorateWindowsBlockedNativeLibraryError(new Error("unrelated failure"), destRoot).message).toBe(
|
||||||
|
"unrelated failure",
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
rmSync(nativeRoot, { recursive: true, force: true });
|
||||||
|
rmSync(destRoot, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("restores a Defender-quarantined dict_snowball.dll on the next launch", () => {
|
||||||
|
const nativeRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-av-src-"));
|
||||||
|
const destRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-av-dst-"));
|
||||||
|
try {
|
||||||
|
seedNativeRoot(nativeRoot, "postgres-stable");
|
||||||
|
materializeEmbeddedPostgresRuntimeBinaries(nativeRoot, { destRoot });
|
||||||
|
rmSync(join(destRoot, "lib", "dict_snowball.dll"));
|
||||||
|
|
||||||
|
materializeEmbeddedPostgresRuntimeBinaries(nativeRoot, { destRoot });
|
||||||
|
expect(readFileSync(join(destRoot, "lib", "dict_snowball.dll"), "utf8")).toBe("snowball-v1");
|
||||||
|
} finally {
|
||||||
|
rmSync(nativeRoot, { recursive: true, force: true });
|
||||||
|
rmSync(destRoot, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("restores a zero-byte truncated postgres executable", () => {
|
||||||
|
const nativeRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-av-bin-src-"));
|
||||||
|
const destRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-av-bin-dst-"));
|
||||||
|
try {
|
||||||
|
seedNativeRoot(nativeRoot, "postgres-stable");
|
||||||
|
materializeEmbeddedPostgresRuntimeBinaries(nativeRoot, { destRoot });
|
||||||
|
writeFileSync(join(destRoot, "bin", postgresBin), "");
|
||||||
|
|
||||||
|
materializeEmbeddedPostgresRuntimeBinaries(nativeRoot, { destRoot });
|
||||||
|
expect(readFileSync(join(destRoot, "bin", postgresBin), "utf8")).toBe("postgres-stable");
|
||||||
|
} finally {
|
||||||
|
rmSync(nativeRoot, { recursive: true, force: true });
|
||||||
|
rmSync(destRoot, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
it("re-copies when payload changes even though nativeRoot path is unchanged", () => {
|
it("re-copies when payload changes even though nativeRoot path is unchanged", () => {
|
||||||
const nativeRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-update-"));
|
const nativeRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-update-"));
|
||||||
const destRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-update-dst-"));
|
const destRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-update-dst-"));
|
||||||
@@ -463,6 +577,29 @@ describe("embedded-lifecycle: materialize runtime binaries (update-safe marker)"
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("treats v2 markers as stale and rematerializes", () => {
|
||||||
|
const nativeRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-v2-"));
|
||||||
|
const destRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-v2-dst-"));
|
||||||
|
try {
|
||||||
|
seedNativeRoot(nativeRoot, "postgres-current");
|
||||||
|
materializeEmbeddedPostgresRuntimeBinaries(nativeRoot, { destRoot });
|
||||||
|
writeFileSync(
|
||||||
|
join(destRoot, ".materialized-from"),
|
||||||
|
buildEmbeddedPostgresMaterializationMarker(nativeRoot).replace("v3\n", "v2\n"),
|
||||||
|
);
|
||||||
|
writeFileSync(join(destRoot, ".reuse-sentinel"), "must-be-removed");
|
||||||
|
|
||||||
|
materializeEmbeddedPostgresRuntimeBinaries(nativeRoot, { destRoot });
|
||||||
|
expect(existsSync(join(destRoot, ".reuse-sentinel"))).toBe(false);
|
||||||
|
expect(readFileSync(join(destRoot, ".materialized-from"), "utf8")).toBe(
|
||||||
|
buildEmbeddedPostgresMaterializationMarker(nativeRoot),
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
rmSync(nativeRoot, { recursive: true, force: true });
|
||||||
|
rmSync(destRoot, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
it("treats legacy path-only markers as stale and rematerializes", () => {
|
it("treats legacy path-only markers as stale and rematerializes", () => {
|
||||||
const nativeRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-legacy-"));
|
const nativeRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-legacy-"));
|
||||||
const destRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-legacy-dst-"));
|
const destRoot = mkdtempSync(join(tmpdir(), "fusion-embedded-mat-legacy-dst-"));
|
||||||
|
|||||||
@@ -32,6 +32,7 @@ import {
|
|||||||
EMBEDDED_PG_ENV,
|
EMBEDDED_PG_ENV,
|
||||||
NO_EMBEDDED_PG_ENV,
|
NO_EMBEDDED_PG_ENV,
|
||||||
resolveStartupDatabaseOptions,
|
resolveStartupDatabaseOptions,
|
||||||
|
formatPostgresSchemaBackendBootError,
|
||||||
} from "../../postgres/startup-factory.js";
|
} from "../../postgres/startup-factory.js";
|
||||||
import { resolveBackend } from "../../postgres/backend-resolver.js";
|
import { resolveBackend } from "../../postgres/backend-resolver.js";
|
||||||
|
|
||||||
@@ -265,6 +266,24 @@ encoding-conversion failure raised when a non-UTF-8 cluster (WIN1252/WIN1254
|
|||||||
from a pre-fix initdb on a non-UTF-8 OS locale) receives the UTF-8 schema
|
from a pre-fix initdb on a non-UTF-8 OS locale) receives the UTF-8 schema
|
||||||
SQL — and nothing else, so ordinary schema errors never delete a data dir.
|
SQL — and nothing else, so ordinary schema errors never delete a data dir.
|
||||||
*/
|
*/
|
||||||
|
describe("startup-factory Windows antivirus boot hint (#3489)", () => {
|
||||||
|
const issueOutput = 'ERROR OUTPUT: FATAL: could not load library "C:/Users/ppp/.fusion/embedded-postgres/runtime-bin/win32-x64/lib/dict_snowball.dll": unknown error 4551';
|
||||||
|
|
||||||
|
it("adds exactly one actionable hint to the wrapped initdb failure", async () => {
|
||||||
|
const message = await formatPostgresSchemaBackendBootError(
|
||||||
|
new Error(`startup-factory: failed to start embedded PostgreSQL: ${issueOutput}`),
|
||||||
|
);
|
||||||
|
expect(message).toMatch(/^startup-factory: failed to initialize PostgreSQL schema backend:/);
|
||||||
|
expect(message).toContain("dict_snowball.dll");
|
||||||
|
expect(message.match(/Windows antivirus blocked a bundled PostgreSQL library/g)).toHaveLength(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not add the antivirus hint to unrelated boot failures", async () => {
|
||||||
|
const message = await formatPostgresSchemaBackendBootError(new Error("connection refused"));
|
||||||
|
expect(message).not.toContain("Windows antivirus blocked a bundled PostgreSQL library");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe("isEncodingConversionError (#2286 recovery trigger)", () => {
|
describe("isEncodingConversionError (#2286 recovery trigger)", () => {
|
||||||
it("matches the encoding-conversion failure from a non-UTF-8 cluster", () => {
|
it("matches the encoding-conversion failure from a non-UTF-8 cluster", () => {
|
||||||
expect(
|
expect(
|
||||||
|
|||||||
@@ -62,7 +62,7 @@ import {
|
|||||||
import { createHash } from "node:crypto";
|
import { createHash } from "node:crypto";
|
||||||
import { homedir } from "node:os";
|
import { homedir } from "node:os";
|
||||||
import { createServer, type Server } from "node:net";
|
import { createServer, type Server } from "node:net";
|
||||||
import { dirname, join, basename, sep } from "node:path";
|
import { dirname, join, basename, sep, resolve } from "node:path";
|
||||||
import { createRequire, syncBuiltinESMExports } from "node:module";
|
import { createRequire, syncBuiltinESMExports } from "node:module";
|
||||||
import { createLogger } from "../process/logger.js";
|
import { createLogger } from "../process/logger.js";
|
||||||
import { redactConnectionString } from "./credential-redact.js";
|
import { redactConnectionString } from "./credential-redact.js";
|
||||||
@@ -202,7 +202,7 @@ const EMBEDDED_PG_BIN_NAMES = new Set([
|
|||||||
* host-local caches always rematerialize after a desktop update that ships a
|
* host-local caches always rematerialize after a desktop update that ships a
|
||||||
* new fingerprinting strategy (e.g. content-hashing lib/share, not path+size).
|
* new fingerprinting strategy (e.g. content-hashing lib/share, not path+size).
|
||||||
*/
|
*/
|
||||||
const MATERIALIZATION_MARKER_VERSION = 2;
|
const MATERIALIZATION_MARKER_VERSION = 3;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* FNXC:DesktopEmbeddedPostgres 2026-07-14-18:30:
|
* FNXC:DesktopEmbeddedPostgres 2026-07-14-18:30:
|
||||||
@@ -361,6 +361,40 @@ function hashPayloadTreeContents(
|
|||||||
* binaries instead of reusing the previous release's host-local cache.
|
* binaries instead of reusing the previous release's host-local cache.
|
||||||
* Legacy path-only markers fail equality and force rematerialization.
|
* Legacy path-only markers fail equality and force rematerialization.
|
||||||
*/
|
*/
|
||||||
|
/*
|
||||||
|
* FNXC:PostgresEmbedded 2026-08-20-01:11:
|
||||||
|
* Issue #3489 reports Windows Defender removing dict_snowball.dll while PostgreSQL
|
||||||
|
* initializes. Win32 4550/4551 are the virus-infected/deleted codes, so match both
|
||||||
|
* signatures narrowly; generic library-load errors have unrelated remediation.
|
||||||
|
*/
|
||||||
|
export function isWindowsBlockedNativeLibraryError(errorOrText: unknown): boolean {
|
||||||
|
const text = errorOrText instanceof Error ? errorOrText.message : String(errorOrText ?? "");
|
||||||
|
return (/could not load library/i.test(text) && /unknown error 455[01]/i.test(text))
|
||||||
|
|| /ERROR_VIRUS_(?:INFECTED|DELETED)/i.test(text);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Creates the shared, actionable Windows antivirus recovery instruction. */
|
||||||
|
export function describeWindowsBlockedNativeLibraryError(errorOrText: unknown): string {
|
||||||
|
const text = errorOrText instanceof Error ? errorOrText.message : String(errorOrText ?? "");
|
||||||
|
const blockedPath = text.match(/could not load library\s+["']([^"']+)["']/i)?.[1];
|
||||||
|
const pathDetail = blockedPath ? ` Blocked file: ${blockedPath}.` : "";
|
||||||
|
return `${pathDetail} HINT: Windows antivirus blocked a bundled PostgreSQL library. Add an exclusion for %USERPROFILE%\\.fusion\\embedded-postgres in Windows Security → Virus & threat protection → Manage settings → Exclusions, restore the quarantined file from Protection history, then restart Fusion. Fusion re-copies the runtime automatically.`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function decorateWindowsBlockedNativeLibraryError(error: unknown, destRoot?: string): Error {
|
||||||
|
const recorded = getEmbeddedPayloadIntegrityFailure(destRoot);
|
||||||
|
if (!isWindowsBlockedNativeLibraryError(error) && !recorded) {
|
||||||
|
return error instanceof Error ? error : new Error(String(error));
|
||||||
|
}
|
||||||
|
const message = error instanceof Error ? error.message : String(error);
|
||||||
|
if (message.includes("Windows antivirus blocked a bundled PostgreSQL library")) {
|
||||||
|
return error instanceof Error ? error : new Error(message);
|
||||||
|
}
|
||||||
|
return new Error(`${message}${recorded ? ` ${recorded.message}` : describeWindowsBlockedNativeLibraryError(error)}`, {
|
||||||
|
cause: error,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
export function buildEmbeddedPostgresMaterializationMarker(nativeRoot: string): string {
|
export function buildEmbeddedPostgresMaterializationMarker(nativeRoot: string): string {
|
||||||
const fingerprint = fingerprintEmbeddedPostgresNativeRoot(nativeRoot);
|
const fingerprint = fingerprintEmbeddedPostgresNativeRoot(nativeRoot);
|
||||||
return `v${MATERIALIZATION_MARKER_VERSION}\n${nativeRoot}\n${fingerprint}\n`;
|
return `v${MATERIALIZATION_MARKER_VERSION}\n${nativeRoot}\n${fingerprint}\n`;
|
||||||
@@ -374,6 +408,134 @@ function resolveMaterializedEmbeddedPostgresBinary(filePath: string): string | n
|
|||||||
return existsSync(candidate) ? candidate : null;
|
return existsSync(candidate) ? candidate : null;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type PayloadInventoryResult = {
|
||||||
|
readonly acceptable: boolean;
|
||||||
|
readonly mismatches: readonly string[];
|
||||||
|
readonly mismatchCount: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* FNXC:PostgresEmbedded 2026-08-20-01:11:
|
||||||
|
* A marker fingerprints only the source. Defender can remove a copied DLL after
|
||||||
|
* materialization, leaving that marker truthful but the destination unusable.
|
||||||
|
* Compare the bounded source inventory on reuse; exhaustion and unreadable source
|
||||||
|
* fail open so a healthy host never incurs an unbounded copy or boot refusal.
|
||||||
|
*/
|
||||||
|
function verifyEmbeddedPostgresPayloadInventory(
|
||||||
|
nativeRoot: string,
|
||||||
|
destRoot: string,
|
||||||
|
): PayloadInventoryResult {
|
||||||
|
const budget = { remaining: 4096 };
|
||||||
|
const mismatches: string[] = [];
|
||||||
|
let mismatchCount = 0;
|
||||||
|
let sourceUnreadable = false;
|
||||||
|
const report = (relativePath: string) => {
|
||||||
|
mismatchCount += 1;
|
||||||
|
if (mismatches.length < 10) mismatches.push(relativePath);
|
||||||
|
};
|
||||||
|
const visit = (sourcePath: string, destPath: string, relativePath: string): void => {
|
||||||
|
if (budget.remaining <= 0 || sourceUnreadable) return;
|
||||||
|
let sourceEntries: string[];
|
||||||
|
try {
|
||||||
|
sourceEntries = readdirSync(sourcePath).sort();
|
||||||
|
} catch {
|
||||||
|
sourceUnreadable = true;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
for (const entry of sourceEntries) {
|
||||||
|
if (budget.remaining <= 0 || sourceUnreadable) return;
|
||||||
|
const sourceEntry = join(sourcePath, entry);
|
||||||
|
const destEntry = join(destPath, entry);
|
||||||
|
const relativeEntry = relativePath ? `${relativePath}/${entry}` : entry;
|
||||||
|
try {
|
||||||
|
const sourceStat = lstatSync(sourceEntry);
|
||||||
|
if (sourceStat.isDirectory()) {
|
||||||
|
visit(sourceEntry, destEntry, relativeEntry);
|
||||||
|
} else if (sourceStat.isFile() || sourceStat.isSymbolicLink()) {
|
||||||
|
budget.remaining -= 1;
|
||||||
|
let destStat: ReturnType<typeof lstatSync>;
|
||||||
|
try {
|
||||||
|
destStat = lstatSync(destEntry);
|
||||||
|
} catch {
|
||||||
|
report(relativeEntry);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (sourceStat.isFile()) {
|
||||||
|
if (!destStat.isFile() || destStat.size !== sourceStat.size) report(relativeEntry);
|
||||||
|
} else {
|
||||||
|
if (!destStat.isSymbolicLink()) report(relativeEntry);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
sourceUnreadable = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
for (const tree of ["bin", "lib", "share"] as const) {
|
||||||
|
const sourceTree = join(nativeRoot, tree);
|
||||||
|
try {
|
||||||
|
if (lstatSync(sourceTree).isDirectory()) visit(sourceTree, join(destRoot, tree), tree);
|
||||||
|
} catch {
|
||||||
|
sourceUnreadable = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
acceptable: sourceUnreadable || budget.remaining <= 0 || mismatchCount === 0,
|
||||||
|
mismatches,
|
||||||
|
mismatchCount,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export class EmbeddedPostgresPayloadBlockedError extends Error {
|
||||||
|
readonly destRoot: string;
|
||||||
|
readonly nativeRoot: string;
|
||||||
|
readonly affectedPaths: readonly string[];
|
||||||
|
readonly affectedPathCount: number;
|
||||||
|
|
||||||
|
constructor(nativeRoot: string, destRoot: string, verification: PayloadInventoryResult) {
|
||||||
|
const affected = verification.mismatches.join(", ") || "unknown payload entry";
|
||||||
|
super(
|
||||||
|
`Embedded PostgreSQL runtime payload is incomplete (${affected}${verification.mismatchCount > verification.mismatches.length ? ", …" : ""}).${describeWindowsBlockedNativeLibraryError(affected)}`,
|
||||||
|
);
|
||||||
|
this.name = "EmbeddedPostgresPayloadBlockedError";
|
||||||
|
this.nativeRoot = nativeRoot;
|
||||||
|
this.destRoot = destRoot;
|
||||||
|
this.affectedPaths = verification.mismatches;
|
||||||
|
this.affectedPathCount = verification.mismatchCount;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type EmbeddedPayloadIntegrityFailure = {
|
||||||
|
readonly destRoot: string;
|
||||||
|
readonly error: EmbeddedPostgresPayloadBlockedError;
|
||||||
|
readonly recordedAt: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
let embeddedPayloadIntegrityFailure: EmbeddedPayloadIntegrityFailure | null = null;
|
||||||
|
|
||||||
|
/** Records only the newest root-scoped incomplete-payload diagnosis. */
|
||||||
|
export function recordEmbeddedPayloadIntegrityFailure(error: EmbeddedPostgresPayloadBlockedError): void {
|
||||||
|
embeddedPayloadIntegrityFailure = { destRoot: resolve(error.destRoot), error, recordedAt: Date.now() };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Returns the diagnostic only for its originating materialized runtime root. */
|
||||||
|
export function getEmbeddedPayloadIntegrityFailure(destRoot?: string): EmbeddedPostgresPayloadBlockedError | null {
|
||||||
|
if (!embeddedPayloadIntegrityFailure) return null;
|
||||||
|
if (destRoot && resolve(destRoot) !== embeddedPayloadIntegrityFailure.destRoot) return null;
|
||||||
|
return embeddedPayloadIntegrityFailure.error;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Clears the test-resettable, in-memory payload diagnostic. */
|
||||||
|
export function clearEmbeddedPayloadIntegrityFailure(): void {
|
||||||
|
embeddedPayloadIntegrityFailure = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function clearEmbeddedPayloadIntegrityFailureForRoot(destRoot: string): void {
|
||||||
|
if (embeddedPayloadIntegrityFailure?.destRoot === resolve(destRoot)) {
|
||||||
|
clearEmbeddedPayloadIntegrityFailure();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export interface MaterializeEmbeddedPostgresOptions {
|
export interface MaterializeEmbeddedPostgresOptions {
|
||||||
/**
|
/**
|
||||||
* Override the host-local dest root (tests). Defaults to
|
* Override the host-local dest root (tests). Defaults to
|
||||||
@@ -413,8 +575,10 @@ export function materializeEmbeddedPostgresRuntimeBinaries(
|
|||||||
existsSync(marker) &&
|
existsSync(marker) &&
|
||||||
readFileSync(marker, "utf8") === sourceMarker &&
|
readFileSync(marker, "utf8") === sourceMarker &&
|
||||||
existsSync(join(destBin, process.platform === "win32" ? "postgres.exe" : "postgres")) &&
|
existsSync(join(destBin, process.platform === "win32" ? "postgres.exe" : "postgres")) &&
|
||||||
existsSync(join(destRoot, "lib", "postgresql"))
|
existsSync(join(destRoot, "lib", "postgresql")) &&
|
||||||
|
verifyEmbeddedPostgresPayloadInventory(nativeRoot, destRoot).acceptable
|
||||||
) {
|
) {
|
||||||
|
clearEmbeddedPayloadIntegrityFailureForRoot(destRoot);
|
||||||
return destRoot;
|
return destRoot;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -424,6 +588,12 @@ export function materializeEmbeddedPostgresRuntimeBinaries(
|
|||||||
* payload cannot linger beside the updated binaries (force-copy alone does not
|
* payload cannot linger beside the updated binaries (force-copy alone does not
|
||||||
* delete orphans).
|
* delete orphans).
|
||||||
*/
|
*/
|
||||||
|
// Never let a prior marker certify a partially copied payload after a crash or AV action.
|
||||||
|
try {
|
||||||
|
unlinkSync(marker);
|
||||||
|
} catch {
|
||||||
|
// best-effort; recursive destination removal below is the normal cleanup path
|
||||||
|
}
|
||||||
if (existsSync(destRoot)) {
|
if (existsSync(destRoot)) {
|
||||||
rmSync(destRoot, { recursive: true, force: true });
|
rmSync(destRoot, { recursive: true, force: true });
|
||||||
}
|
}
|
||||||
@@ -446,7 +616,19 @@ export function materializeEmbeddedPostgresRuntimeBinaries(
|
|||||||
}
|
}
|
||||||
// Re-apply macOS ABI compatibility links against the materialized lib dir.
|
// Re-apply macOS ABI compatibility links against the materialized lib dir.
|
||||||
normalizeMacosEmbeddedPostgresDylibSymlinks(destRoot);
|
normalizeMacosEmbeddedPostgresDylibSymlinks(destRoot);
|
||||||
|
const verification = verifyEmbeddedPostgresPayloadInventory(nativeRoot, destRoot);
|
||||||
|
if (!verification.acceptable) {
|
||||||
|
try {
|
||||||
|
unlinkSync(marker);
|
||||||
|
} catch {
|
||||||
|
// No marker is the retry contract; ignore an already-absent marker.
|
||||||
|
}
|
||||||
|
const error = new EmbeddedPostgresPayloadBlockedError(nativeRoot, destRoot, verification);
|
||||||
|
recordEmbeddedPayloadIntegrityFailure(error);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
writeFileSync(marker, sourceMarker, "utf8");
|
writeFileSync(marker, sourceMarker, "utf8");
|
||||||
|
clearEmbeddedPayloadIntegrityFailureForRoot(destRoot);
|
||||||
return destRoot;
|
return destRoot;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -525,7 +707,10 @@ export function installElectronAsarNativePathPatch(): void {
|
|||||||
materializeEmbeddedPostgresRuntimeBinaries(sourceRoot);
|
materializeEmbeddedPostgresRuntimeBinaries(sourceRoot);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch {
|
} catch (error) {
|
||||||
|
if (error instanceof EmbeddedPostgresPayloadBlockedError) {
|
||||||
|
recordEmbeddedPayloadIntegrityFailure(error);
|
||||||
|
}
|
||||||
// Materialization is best-effort; path rewrite still helps when possible.
|
// Materialization is best-effort; path rewrite still helps when possible.
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -970,7 +1155,10 @@ function resolveWindowsEmbeddedPostgresNativeRoot(): string | null {
|
|||||||
return materializeEmbeddedPostgresRuntimeBinaries(
|
return materializeEmbeddedPostgresRuntimeBinaries(
|
||||||
resolveElectronAsarUnpackedPath(nativeRoot),
|
resolveElectronAsarUnpackedPath(nativeRoot),
|
||||||
);
|
);
|
||||||
} catch {
|
} catch (error) {
|
||||||
|
if (error instanceof EmbeddedPostgresPayloadBlockedError) {
|
||||||
|
recordEmbeddedPayloadIntegrityFailure(error);
|
||||||
|
}
|
||||||
return resolveElectronAsarUnpackedPath(nativeRoot);
|
return resolveElectronAsarUnpackedPath(nativeRoot);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1531,7 +1719,11 @@ export class EmbeddedPostgresLifecycle {
|
|||||||
this.options.onLog(
|
this.options.onLog(
|
||||||
`embedded postgres: initializing new data directory at ${this.options.dataDir} (initdb)`,
|
`embedded postgres: initializing new data directory at ${this.options.dataDir} (initdb)`,
|
||||||
);
|
);
|
||||||
await pg.initialise();
|
try {
|
||||||
|
await pg.initialise();
|
||||||
|
} catch (error) {
|
||||||
|
throw decorateWindowsBlockedNativeLibraryError(error, embeddedPostgresRuntimeBinRoot());
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (signal?.aborted) {
|
if (signal?.aborted) {
|
||||||
@@ -1603,7 +1795,9 @@ export class EmbeddedPostgresLifecycle {
|
|||||||
then failed later read back its OWN postmaster.pid, "join itself" with ownsProcess=false,
|
then failed later read back its OWN postmaster.pid, "join itself" with ownsProcess=false,
|
||||||
and orphan a live postmaster nothing would ever stop. See isPostgresLockCollisionError.
|
and orphan a live postmaster nothing would ever stop. See isPostgresLockCollisionError.
|
||||||
*/
|
*/
|
||||||
if (!isPostgresLockCollisionError(error)) throw error;
|
if (!isPostgresLockCollisionError(error)) {
|
||||||
|
throw decorateWindowsBlockedNativeLibraryError(error, embeddedPostgresRuntimeBinRoot());
|
||||||
|
}
|
||||||
const existing = await isAlreadyRunning(this.options.dataDir, this.options.onLog);
|
const existing = await isAlreadyRunning(this.options.dataDir, this.options.onLog);
|
||||||
if (!existing) throw error;
|
if (!existing) throw error;
|
||||||
|
|
||||||
|
|||||||
@@ -130,6 +130,19 @@ function fallbackProjectIdForRoot(rootDir: string): string {
|
|||||||
* kept for backward compatibility with scripts/docs that still set it. It
|
* kept for backward compatibility with scripts/docs that still set it. It
|
||||||
* cannot force embedded mode when DATABASE_URL is set (external always wins).
|
* cannot force embedded mode when DATABASE_URL is set (external always wins).
|
||||||
*/
|
*/
|
||||||
|
export async function formatPostgresSchemaBackendBootError(err: unknown): Promise<string> {
|
||||||
|
const chain = describeErrorChain(err);
|
||||||
|
const encodingHint = /has no equivalent in encoding/i.test(chain)
|
||||||
|
? " HINT: this embedded PostgreSQL cluster was created with a non-UTF-8 encoding inherited from the OS locale by an earlier Fusion version. It cannot be converted in place — stop Fusion, delete the embedded data directory (default: ~/.fusion/embedded-postgres/default), and start again so the cluster is recreated as UTF-8."
|
||||||
|
: "";
|
||||||
|
const { isWindowsBlockedNativeLibraryError, describeWindowsBlockedNativeLibraryError } =
|
||||||
|
await import("./embedded-lifecycle.js");
|
||||||
|
const blockedLibraryHint = isWindowsBlockedNativeLibraryError(chain)
|
||||||
|
? describeWindowsBlockedNativeLibraryError(chain)
|
||||||
|
: "";
|
||||||
|
return `startup-factory: failed to initialize PostgreSQL schema backend: ${chain}${encodingHint}${blockedLibraryHint}`;
|
||||||
|
}
|
||||||
|
|
||||||
export const EMBEDDED_PG_ENV = "FUSION_EMBEDDED_PG";
|
export const EMBEDDED_PG_ENV = "FUSION_EMBEDDED_PG";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -869,20 +882,13 @@ export async function createTaskStoreForBackend(
|
|||||||
boot = await bootSchemaBackend(effectiveOptions);
|
boot = await bootSchemaBackend(effectiveOptions);
|
||||||
log.log(`startup phase backend.schemaBackend: ${Date.now() - schemaT0}ms`);
|
log.log(`startup phase backend.schemaBackend: ${Date.now() - schemaT0}ms`);
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const chain = describeErrorChain(err);
|
|
||||||
/*
|
/*
|
||||||
FNXC:PostgresEmbedded 2026-07-18-00:20:
|
FNXC:PostgresEmbedded 2026-08-20-01:11:
|
||||||
Issue #2286: a cluster initdb'd by an earlier version on a non-UTF-8 OS
|
Issue #3489 uses the same outer boot mapper as #2286's encoding guidance.
|
||||||
locale cannot store the UTF-8 schema SQL and cannot be converted in place.
|
initdb wraps PostgreSQL's FATAL before this boundary, so retain the chain
|
||||||
Newly created clusters are forced to UTF-8 (DEFAULT_EMBEDDED_INITDB_FLAGS);
|
while adding one actionable antivirus hint only for 4550/4551 signatures.
|
||||||
existing ones need a manual re-init, so say exactly that.
|
|
||||||
*/
|
*/
|
||||||
const encodingHint = /has no equivalent in encoding/i.test(chain)
|
throw new Error(await formatPostgresSchemaBackendBootError(err));
|
||||||
? " HINT: this embedded PostgreSQL cluster was created with a non-UTF-8 encoding inherited from the OS locale by an earlier Fusion version. It cannot be converted in place — stop Fusion, delete the embedded data directory (default: ~/.fusion/embedded-postgres/default), and start again so the cluster is recreated as UTF-8."
|
|
||||||
: "";
|
|
||||||
throw new Error(
|
|
||||||
`startup-factory: failed to initialize PostgreSQL schema backend: ${chain}${encodingHint}`,
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
let { connections } = boot;
|
let { connections } = boot;
|
||||||
const {
|
const {
|
||||||
|
|||||||
Reference in New Issue
Block a user