FN-078: repair multi-repository workspace merge coordination
Keep long-running workspace land operations live and enforce repository-local scope isolation. - Renew workspace land leases throughout dependency sync, AI merge, push, and intent resolution. - Abort stale land bodies before durable or remote commit points and fence lease-dependent writes. - Centralize repository File Scope resolution across squash, completion, and checkout guards. - Add regression coverage, architecture documentation, and a patch changeset. Files changed: .changeset/fn-078-workspace-lease-and-scope.md | 7 + docs/architecture.md | 3 +- .../executor-workspace-main-checkout-guard.test.ts | 13 ++ .../__tests__/executor-workspace-taskdone.test.ts | 21 +++ .../src/__tests__/workspace-merger-lease.test.ts | 195 ++++++++++++++++++++- .../src/executor/workspace-main-checkout-guard.ts | 4 +- .../src/executor/worktree-task-done-scope-leak.ts | 4 +- .../engine/src/merge/merger-ai-squash-gates.ts | 9 +- packages/engine/src/merge/merger-ai.ts | 55 +++++- packages/engine/src/worktree/workspace-paths.ts | 26 +++ 10 files changed, 321 insertions(+), 16 deletions(-) Fusion-Task-Id: FN-078 Fusion-Task-Lineage: 8ddfa438-d6c0-4d3a-972a-3594bfc05ffb Co-authored-by: Fusion <noreply@runfusion.ai>
This commit is contained in:
Fusion Agent
7
.changeset/fn-078-workspace-lease-and-scope.md
Normal file
7
.changeset/fn-078-workspace-lease-and-scope.md
Normal file
@@ -0,0 +1,7 @@
|
||||
---
|
||||
"@runfusion/fusion": patch
|
||||
---
|
||||
|
||||
summary: Keep multi-repository merges live during long AI land operations.
|
||||
category: fix
|
||||
dev: Renews repository land leases and unifies workspace File Scope resolution across merge and completion guards.
|
||||
@@ -630,7 +630,8 @@ See [Memory Plugin Contract](./memory-plugin-contract.md) for the full plan.
|
||||
- **No root task worktree (FN-034):** when `.fusion/workspace.json` declares repositories, planning, execution, graph nodes, code/plan review, pause/resume, restart, and worktree-session recovery acquire or reuse only the declared repositories' worktrees. The workspace root and `<workspace>/.worktrees/<task>` are never task checkout or reviewer cwd. Historical singular `worktree`, `branch`, execution-base, and root-session routing metadata is cleared through an advisory-locked, project-scoped mutation that preserves every `workspaceWorktrees` entry and completed-step progress; recovery never deletes or recreates a root checkout.
|
||||
- **Workspace entry mutation (FN-9052):** every per-repository `workspaceWorktrees` update goes through `TaskStore.mergeWorkspaceWorktreeEntry`, which holds the per-task PostgreSQL advisory transaction lock and merges one key under the composite project/task scope. Per-repo callers must never wholesale-replace the map, because a concurrent sibling acquisition, landing, failure, or teardown mutation would otherwise lose its entry.
|
||||
- **Durable multi-node workspace coordination (FN-9059):** `project.workspace_coordination_leases`, serialized under a project/resource advisory transaction lock, owns sub-repository acquire exclusivity, per-repository land, workspace liveness/recovery guards, and merge-dispatch admission across engine processes. The owner triple `(taskId, nodeId, incarnationId)` makes only same-process re-entry idempotent; the same task from another node or process incarnation contends. Fresh/reclaimed claims receive a monotonically increasing fence token, while re-entry and renewal retain it. Git-writing tenancies publish once per tenancy—on acquire/reclaimed-expired, not re-entry—the per-repository `refs/fusion/workspace-lease/<repo-slug>` or per-merge `refs/fusion/merge-dispatch/<task-id>` fence ref. A workspace merge dispatches its deterministic per-merge pin to every target sub-repository remote before any workspace land begins; the workspace root need not be a git checkout and is never used as a substitute remote. A re-entrant claimant reuses its pin, except to repair a claim-to-publish gap with no pin; acquire-kind leases have no fence ref. A land push atomically CASes the target ref observed by the tenancy plus its repository fence and enclosing merge-dispatch fence refs in one push; a merge-only push CASes its target and dispatch pin. Thus a superseded owner is rejected even when the target tip has not moved. Lease-protected durable writes run through `withValidWorkspaceLeaseAsync`, and lease mutation is owner-and-fence conditional; no validate-then-act path is valid.
|
||||
- **Workspace merge and land crash boundaries (FN-9059):** merge dispatch claims at body dispatch rather than enqueue; a losing claimant benignly drops. The body re-proves its fence at admission, the target-plus-fence atomic push, subsequent PR/branch/status effects, and terminal outcome persistence. Non-CASable effects are idempotent and follow the fenced push, so mid-merge expiry stops a superseded body at its next commit point; a pushed result whose outcome cannot persist is `merge-completed-unrecorded`, never a re-push. Before workspace land pushes, `project.workspace_land_intents` records the expected tip, intended SHA, remote/integration identity, and fence pin. The ordered protocol is intent → atomic push → lease-validated `landedSha` persist plus intent resolution. The node-independent reconciler lists pending intents project-wide, fetches the recorded remote, and resolves reachability on its integration ref. Only a live holder (its own/equal fence or a lower-fence predecessor) or recovery with no held unexpired lease can resolve an intent; stale fence matches, local tip equality, and local object availability are not authority.
|
||||
- **Workspace merge and land crash boundaries (FN-9059):** merge dispatch claims at body dispatch rather than enqueue; a losing claimant benignly drops. Each live per-repository land lease renews before its five-minute TTL throughout dependency sync, AI merge/review, retry, push, and intent resolution. A refused or failed renewal aborts the land body before its next durable or remote commit point; renewal only maintains liveness, while the current owner/fence handle remains the write authority. The body re-proves its fence at admission, the target-plus-fence atomic push, subsequent PR/branch/status effects, and terminal outcome persistence. Non-CASable effects are idempotent and follow the fenced push, so mid-merge expiry stops a superseded body at its next commit point; a pushed result whose outcome cannot persist is `merge-completed-unrecorded`, never a re-push. Before workspace land pushes, `project.workspace_land_intents` records the expected tip, intended SHA, remote/integration identity, and fence pin. The ordered protocol is intent → atomic push → lease-validated `landedSha` persist plus intent resolution. The node-independent reconciler lists pending intents project-wide, fetches the recorded remote, and resolves reachability on its integration ref. Only a live holder (its own/equal fence or a lower-fence predecessor) or recovery with no held unexpired lease can resolve an intent; stale fence matches, local tip equality, and local object availability are not authority.
|
||||
- **Workspace File Scope interpretation (FN-078):** squash gates, completion scope-leak checks, and main-checkout guards use one repo-local resolver. A matching repo-prefixed declaration has priority; only a declaration with no configured repository prefix can fall back as repo-local scope. If any declaration belongs to another configured repository, a repo with no matching prefix receives no fallback authority. This preserves whole-repo `repo` → `**`, normalized segment-aware nested keys, empty-scope behavior, and strict foreign-repository isolation.
|
||||
- **Main-checkout completion guard (FN-9058):** `fn_task_done` probes every configured sub-repo main checkout before any workspace worktree invariant, so `main_checkout_edit` takes precedence over `no_commits` and cannot be skipped by zero-acquire or no-commit eligibility. It uses the immutable first-execution anchor (never only the re-stamped per-attempt timestamp), blocks task-era status entries and bounded recent-HEAD evidence without `--since` or ancestry filtering, and emits `worktree:workspace-main-checkout-edit`. Unattributable pre-existing dirt, unavailable probes, and unresolved timing only warn: refusal has a bounded requeue budget and the guard is read-only.
|
||||
- **Task-pinned orphan recovery:** task-ID-pinned acquisition holds one path reservation across classification, preservation, quarantine reconciliation, and recreation. Inactive incomplete or unregistered directories are atomically moved to `<project>/.fusion/recovery/worktrees`, or to `<worktreesDir>/.fusion-recovery/worktrees` after an `EXDEV` cross-filesystem refusal. Each actual recovery root retains the newest 10 recognized Fusion-generated entries; pruning is fail-soft and preserves unknown, symlinked, unreadable, or active paths. Worktree pool and self-healing scans exclude both `.ai-merge` and `.fusion-recovery` as internal container boundaries.
|
||||
<!-- FNXC:MergerUnification 2026-08-09-12:04: Master-plan U0 made clean-room `runAiMerge` the sole production merge path. The legacy `aiMergeTask` auto-prerebase policy is retained but inert, so executor reused-base refresh must not describe it as live merger behavior. -->
|
||||
|
||||
@@ -62,6 +62,19 @@ describeIfGit("workspace main-checkout guard", () => {
|
||||
expect(result.violations.find((finding) => finding.repo === "repo-b")?.files).toContain("src/new.ts");
|
||||
});
|
||||
|
||||
it("treats repo-local File Scope as declared scope for a single workspace repository", async () => {
|
||||
fixture = await createWorkspaceFixture(["repo-a"]);
|
||||
const activeTask = task();
|
||||
const file = path.join(fixture.repoPath("repo-a"), "src", "local.ts");
|
||||
mkdirSync(path.dirname(file), { recursive: true });
|
||||
writeFileSync(file, "export {};\n");
|
||||
const changed = new Date(Date.parse(activeTask.firstExecutionAt!) + 10_000);
|
||||
await import("node:fs/promises").then(({ utimes }) => utimes(file, changed, changed));
|
||||
|
||||
const result = await detectWorkspaceMainCheckoutWork({ rootDir: fixture.rootDir, settings }, activeTask, fixture.repos, ["src/**"]);
|
||||
expect(result.violations).toContainEqual(expect.objectContaining({ repo: "repo-a", files: ["src/local.ts"], evidence: "declared-scope-change" }));
|
||||
});
|
||||
|
||||
it("uses firstExecutionAt instead of the later retry attempt anchor", async () => {
|
||||
fixture = await createWorkspaceFixture(["repo-a"]);
|
||||
const first = new Date(Date.now() + 1_000).toISOString();
|
||||
|
||||
@@ -25,6 +25,7 @@ import { TaskExecutor } from "../executor.js";
|
||||
import {
|
||||
deriveRepoForPath,
|
||||
deriveRepoScopeSubset,
|
||||
resolveRepoDeclaredScope,
|
||||
splitRepoScopedPath,
|
||||
UNSCOPED_REPO,
|
||||
} from "../worktree/workspace-paths.js";
|
||||
@@ -115,6 +116,13 @@ describe("U2 — workspace-paths repo-prefix helper (unit)", () => {
|
||||
// repo-root scope entry maps to whole-repo **
|
||||
expect(deriveRepoScopeSubset(["repo-a"], "repo-a")).toEqual(["**"]);
|
||||
});
|
||||
it("resolves prefixed, repo-local, and foreign-only scope without cross-repo fallback", () => {
|
||||
const repoKeys = ["repo-a", "apps/web", "repo-b"];
|
||||
expect(resolveRepoDeclaredScope(["apps/web/src/**", "repo-a"], "apps/web", repoKeys)).toEqual({ scope: ["src/**"], source: "repo-subset" });
|
||||
expect(resolveRepoDeclaredScope(["src/**", "README.md"], "repo-a", repoKeys)).toEqual({ scope: ["src/**", "README.md"], source: "unprefixed-fallback" });
|
||||
expect(resolveRepoDeclaredScope(["repo-a/src/**", "src/**"], "repo-b", repoKeys)).toEqual({ scope: [], source: "foreign-repo-only" });
|
||||
expect(resolveRepoDeclaredScope(["repo-a", "repo-a/src/**"], "repo-a", repoKeys)).toEqual({ scope: ["**", "src/**"], source: "repo-subset" });
|
||||
});
|
||||
});
|
||||
|
||||
describeIfGit("U2 KTD4 — per-repo scope-leak guard in fn_task_done", () => {
|
||||
@@ -147,6 +155,19 @@ describeIfGit("U2 KTD4 — per-repo scope-leak guard in fn_task_done", () => {
|
||||
expect(result.message).toContain("OFFSCOPE.md");
|
||||
});
|
||||
|
||||
it("accepts repo-local scope for a one-repository workspace", async () => {
|
||||
fx = await createWorkspaceFixture(["repo-a"]);
|
||||
const a = addRepoWorktree(fx, "repo-a", "src/a.ts");
|
||||
const store = createStore(["src/**"]);
|
||||
const executor = workspaceExecutor(fx, store);
|
||||
const task = makeTask({
|
||||
branch: BRANCH,
|
||||
workspaceWorktrees: { "repo-a": { worktreePath: a.worktreePath, branch: BRANCH, baseCommitSha: a.baseCommitSha } },
|
||||
});
|
||||
|
||||
await expect((executor as any).evaluateTaskDoneScopeLeak(task, fx.rootDir, PROMPT, SETTINGS)).resolves.toEqual({ blocked: false });
|
||||
});
|
||||
|
||||
it("all-clean: only in-scope changes in both repos → not blocked", async () => {
|
||||
fx = await createWorkspaceFixture();
|
||||
const a = addRepoWorktree(fx, "repo-a", "src/a.ts");
|
||||
|
||||
@@ -26,7 +26,7 @@ import { EventEmitter } from "node:events";
|
||||
import { execSync } from "node:child_process";
|
||||
import { writeFileSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import type { Task, TaskStore } from "@fusion/core";
|
||||
import type { Task, TaskStore, WorkspaceLeaseHandle } from "@fusion/core";
|
||||
import { createSharedPgTaskStoreTestHarness, pgDescribe, type SharedPgTaskStoreHarness } from "../../../core/src/__test-utils__/pg-test-harness.js";
|
||||
import { landSquash, landWorkspaceTask, WorkspaceMergeDispatchSupersededError, WorkspaceRepoLandBusyError } from "../merge/merger-ai.js";
|
||||
import { ensureTenancyFenceRef, mergeDispatchFenceRef, WorkspaceFenceRefError } from "../merge/workspace-fence-ref.js";
|
||||
@@ -232,6 +232,88 @@ pgDescribeIfGit("workspace land dispatch finalization (PostgreSQL)", () => {
|
||||
expect(fx.git("repo-b", `git ls-remote origin ${mergeDispatchFenceRef(taskId)}`)).toMatch(/^[0-9a-f]{40,64}\s/);
|
||||
});
|
||||
|
||||
/*
|
||||
FNXC:Workspace 2026-08-20-20:08:
|
||||
A per-repository lander must lose every post-loss commit point when a durable successor
|
||||
reclaims its expired repo lease. Exercise that ownership change through PostgreSQL rather
|
||||
than a false-return mock, so the predecessor cannot write intent, push, persist landedSha,
|
||||
or release the successor's handle.
|
||||
*/
|
||||
it("fences a real repository lander after a successor reclaims its durable repo lease", async () => {
|
||||
const store = h.store();
|
||||
const taskId = "FN-078-PG-REPO-TAKEOVER";
|
||||
const repoRel = "repo-a";
|
||||
const repo = fx.repoPath(repoRel);
|
||||
const remote = path.join(fx.rootDir, "origin.git");
|
||||
execSync(`git init --bare ${remote}`, { stdio: "pipe" });
|
||||
fx.git(repoRel, `git remote add origin ${remote}`);
|
||||
fx.git(repoRel, "git push -u origin main");
|
||||
addRepoBranchWithEdit(fx, repoRel, taskId, "predecessor must be fenced\n");
|
||||
|
||||
await store.createTaskWithReservedId(
|
||||
{ description: "repository lease successor takeover", column: "in-review" },
|
||||
{ taskId, applyDefaultWorkflowSteps: false },
|
||||
);
|
||||
await store.updateTask(taskId, {
|
||||
branch: BRANCH,
|
||||
workspaceWorktrees: { [repoRel]: { worktreePath: repo, branch: BRANCH } },
|
||||
} as Partial<Task>);
|
||||
const task = (await store.getTask(taskId))!;
|
||||
const tipBefore = fx.git(repoRel, "git rev-parse main");
|
||||
const realRenew = store.renewWorkspaceLease.bind(store);
|
||||
const recordIntent = vi.spyOn(store, "recordWorkspaceLandIntent");
|
||||
const resolveIntent = vi.spyOn(store, "resolveWorkspaceLandIntent");
|
||||
const releaseLease = vi.spyOn(store, "releaseWorkspaceLease");
|
||||
let successorHandle: WorkspaceLeaseHandle | undefined;
|
||||
let successorClaimed = false;
|
||||
|
||||
vi.spyOn(store, "renewWorkspaceLease").mockImplementation(async (handle, leaseMs) => {
|
||||
if (!successorClaimed) {
|
||||
await h.adminSql().unsafe(`UPDATE project.workspace_coordination_leases
|
||||
SET expires_at = '${new Date(Date.now() - 1_000).toISOString()}'
|
||||
WHERE lease_key = '${handle.leaseKey}'`);
|
||||
const successor = await store.acquireWorkspaceLease({
|
||||
leaseKey: handle.leaseKey,
|
||||
kind: "land",
|
||||
owner: { taskId: "FN-078-PG-SUCCESSOR", nodeId: "node-successor", incarnationId: "inc-successor" },
|
||||
leaseMs,
|
||||
});
|
||||
expect(successor.outcome).toBe("reclaimed-expired");
|
||||
if (successor.outcome === "conflict") throw new Error("expected successor repo lease claim");
|
||||
successorHandle = successor.handle;
|
||||
successorClaimed = true;
|
||||
}
|
||||
return realRenew(handle, leaseMs);
|
||||
});
|
||||
|
||||
vi.useFakeTimers();
|
||||
await expect(landWorkspaceTask(store, task, fx.rootDir, {}, {
|
||||
mergeAgent: squashMergeAgent(BRANCH, async () => {
|
||||
await vi.advanceTimersByTimeAsync(60_000);
|
||||
}),
|
||||
reviewAgent: approveReviewAgent,
|
||||
})).rejects.toBeInstanceOf(WorkspaceRepoLandBusyError);
|
||||
|
||||
expect(successorClaimed).toBe(true);
|
||||
expect(successorHandle).toBeDefined();
|
||||
const successor = successorHandle!;
|
||||
expect(recordIntent).not.toHaveBeenCalled();
|
||||
expect(resolveIntent).not.toHaveBeenCalled();
|
||||
expect(fx.git(repoRel, "git rev-parse main")).toBe(tipBefore);
|
||||
const [heldLease] = await store.inspectWorkspaceLeases({ leaseKeys: [`repo:${repoRel}`] });
|
||||
expect(heldLease).toMatchObject({
|
||||
status: "held",
|
||||
owner: successor.owner,
|
||||
fenceToken: successor.fenceToken,
|
||||
});
|
||||
expect(releaseLease).toHaveBeenCalledWith(expect.objectContaining({
|
||||
owner: expect.not.objectContaining(successor.owner),
|
||||
fenceToken: expect.not.toBe(successor.fenceToken),
|
||||
}));
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it("leaves a real pushed land unfinalized when a successor reclaims the dispatch fence", async () => {
|
||||
const store = h.store();
|
||||
const taskId = "FN-9059-PG-DISPATCH";
|
||||
@@ -307,6 +389,7 @@ pgDescribeIfGit("workspace land dispatch finalization (PostgreSQL)", () => {
|
||||
describeIfGit("landWorkspaceTask — per-repo land lease (Phase C U3, KTD4)", () => {
|
||||
let fx: WorkspaceFixture;
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
fx?.cleanup();
|
||||
activeSessionRegistry.clear();
|
||||
vi.restoreAllMocks();
|
||||
@@ -450,6 +533,116 @@ describeIfGit("landWorkspaceTask — per-repo land lease (Phase C U3, KTD4)", ()
|
||||
expect(fx.git("repo-a", "git rev-parse main")).not.toBe(tipBefore);
|
||||
});
|
||||
|
||||
/*
|
||||
FNXC:Workspace 2026-08-20-19:59:
|
||||
Repository land renewal is a liveness aid, not a substitute for the durable owner/fence check.
|
||||
These real-Git tests force the periodic seam while a clean-room merge is active so an expired or
|
||||
successor-reclaimed owner cannot create an intent, push, or terminal workspace mutation.
|
||||
*/
|
||||
it("renews a repository land lease through the former TTL before intent and landed persistence", async () => {
|
||||
fx = await createWorkspaceFixture(["repo-a"]);
|
||||
const repoRel = "repo-a";
|
||||
const repo = fx.repoPath(repoRel);
|
||||
const remote = path.join(fx.rootDir, "origin.git");
|
||||
execSync(`git init --bare ${remote}`, { stdio: "pipe" });
|
||||
fx.git(repoRel, `git remote add origin ${remote}`);
|
||||
fx.git(repoRel, "git push -u origin main");
|
||||
addRepoBranchWithEdit(fx, repoRel, "FN-078-RENEW", "renewed land\n");
|
||||
const task = makeTask("FN-078-RENEW", { [repoRel]: { worktreePath: repo, branch: BRANCH } });
|
||||
const store = createStore(task);
|
||||
const initialHandle = {
|
||||
leaseKey: `repo:${repoRel}`, kind: "land" as const,
|
||||
owner: { taskId: task.id, nodeId: "node-a", incarnationId: "inc-a" }, fenceToken: 1n,
|
||||
};
|
||||
let latestHandle = initialHandle;
|
||||
const renewWorkspaceLease = vi.fn(async (handle: typeof initialHandle) => {
|
||||
latestHandle = { ...handle };
|
||||
return latestHandle;
|
||||
});
|
||||
const resolveWorkspaceLandIntent = vi.fn(async (input: any) => {
|
||||
expect(input.handle).toBe(latestHandle);
|
||||
await input.persistLandedSha();
|
||||
return { outcome: "resolved" };
|
||||
});
|
||||
Object.assign(store, {
|
||||
acquireWorkspaceLease: vi.fn().mockResolvedValue({ outcome: "acquired", handle: initialHandle }),
|
||||
renewWorkspaceLease,
|
||||
recordWorkspaceLeaseFenceRef: vi.fn(async (input: any) => ({ ...input.handle, fenceRefName: input.fenceRefName, fenceRefSha: input.fenceRefSha })),
|
||||
recordWorkspaceLandIntent: vi.fn().mockResolvedValue({ outcome: "valid" }),
|
||||
resolveWorkspaceLandIntent,
|
||||
releaseWorkspaceLease: vi.fn().mockResolvedValue(true),
|
||||
});
|
||||
vi.useFakeTimers();
|
||||
const result = await landWorkspaceTask(store, task, fx.rootDir, {}, {
|
||||
mergeAgent: squashMergeAgent(BRANCH, async () => {
|
||||
// Advance beyond the incident's five-minute TTL while the land critical section is live.
|
||||
await vi.advanceTimersByTimeAsync(5 * 60_000);
|
||||
}),
|
||||
reviewAgent: approveReviewAgent,
|
||||
});
|
||||
|
||||
expect(result.repos[0]?.status).toBe("landed");
|
||||
expect(renewWorkspaceLease).toHaveBeenCalledTimes(5);
|
||||
expect(resolveWorkspaceLandIntent).toHaveBeenCalledOnce();
|
||||
expect((store.releaseWorkspaceLease as any).mock.calls[0][0]).toBe(latestHandle);
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it.each([
|
||||
["returns no renewed handle", async () => undefined],
|
||||
["throws while renewing", async () => { throw new Error("renewal transport failure"); }],
|
||||
])("fences a stale repository lander when renewal %s", async (_description, renew) => {
|
||||
fx = await createWorkspaceFixture(["repo-a"]);
|
||||
const repoRel = "repo-a";
|
||||
const repo = fx.repoPath(repoRel);
|
||||
const remote = path.join(fx.rootDir, "origin.git");
|
||||
execSync(`git init --bare ${remote}`, { stdio: "pipe" });
|
||||
fx.git(repoRel, `git remote add origin ${remote}`);
|
||||
fx.git(repoRel, "git push -u origin main");
|
||||
addRepoBranchWithEdit(fx, repoRel, "FN-078-LOSS", "stale land\n");
|
||||
const tipBefore = fx.git(repoRel, "git rev-parse main");
|
||||
const task = makeTask("FN-078-LOSS", { [repoRel]: { worktreePath: repo, branch: BRANCH } });
|
||||
const store = createStore(task);
|
||||
const predecessorHandle = {
|
||||
leaseKey: `repo:${repoRel}`, kind: "land" as const,
|
||||
owner: { taskId: task.id, nodeId: "node-a", incarnationId: "inc-a" }, fenceToken: 1n,
|
||||
};
|
||||
const successorHandle = { ...predecessorHandle, owner: { taskId: "FN-078-SUCCESSOR", nodeId: "node-b", incarnationId: "inc-b" }, fenceToken: 2n };
|
||||
const recordWorkspaceLandIntent = vi.fn().mockResolvedValue({ outcome: "valid" });
|
||||
Object.assign(store, {
|
||||
acquireWorkspaceLease: vi.fn().mockResolvedValue({ outcome: "acquired", handle: predecessorHandle }),
|
||||
renewWorkspaceLease: vi.fn(renew),
|
||||
recordWorkspaceLeaseFenceRef: vi.fn(async (input: any) => ({ ...input.handle, fenceRefName: input.fenceRefName, fenceRefSha: input.fenceRefSha })),
|
||||
recordWorkspaceLandIntent,
|
||||
resolveWorkspaceLandIntent: vi.fn(),
|
||||
releaseWorkspaceLease: vi.fn().mockResolvedValue(true),
|
||||
});
|
||||
vi.useFakeTimers();
|
||||
await expect(landWorkspaceTask(store, task, fx.rootDir, {}, {
|
||||
mergeAgent: squashMergeAgent(BRANCH, async () => {
|
||||
// The renewal refusal models a successor reclaiming the expired durable handle.
|
||||
await vi.advanceTimersByTimeAsync(60_000);
|
||||
}),
|
||||
reviewAgent: approveReviewAgent,
|
||||
})).rejects.toBeInstanceOf(WorkspaceRepoLandBusyError);
|
||||
|
||||
expect(recordWorkspaceLandIntent).not.toHaveBeenCalled();
|
||||
expect((store.resolveWorkspaceLandIntent as any)).not.toHaveBeenCalled();
|
||||
expect(fx.git(repoRel, "git rev-parse main")).toBe(tipBefore);
|
||||
// Release carries the predecessor identity/fence and can never release the successor's claim.
|
||||
expect((store.releaseWorkspaceLease as any).mock.calls[0][0]).toMatchObject({
|
||||
owner: predecessorHandle.owner,
|
||||
fenceToken: predecessorHandle.fenceToken,
|
||||
});
|
||||
expect((store.releaseWorkspaceLease as any).mock.calls[0][0]).not.toMatchObject({
|
||||
owner: successorHandle.owner,
|
||||
fenceToken: successorHandle.fenceToken,
|
||||
});
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it("concurrency: two tasks landing the SAME sub-repo serialize — one acquires the land lease, the other fast-fails (no interleaved update-ref)", async () => {
|
||||
fx = await createWorkspaceFixture(["repo-a"]);
|
||||
addRepoBranchWithEdit(fx, "repo-a", "FN-3001", "a feature\n");
|
||||
|
||||
@@ -9,7 +9,7 @@ import { existsSync, promises as fs } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import type { Settings, Task } from "@fusion/core";
|
||||
import { deriveRepoScopeSubset, normalizeRepoRelPath } from "../worktree/workspace-paths.js";
|
||||
import { normalizeRepoRelPath, resolveRepoDeclaredScope } from "../worktree/workspace-paths.js";
|
||||
import { resolveWorktreesDir } from "../worktree/worktree-paths.js";
|
||||
import { isAlwaysAllowedScopeLeakPath, workflowPathMatchesDeclaredScope } from "./workflow-feedback-paths.js";
|
||||
|
||||
@@ -92,7 +92,7 @@ export async function detectWorkspaceMainCheckoutWork(
|
||||
continue;
|
||||
}
|
||||
} catch { skipped.push(repo); continue; }
|
||||
const repoScope = deriveRepoScopeSubset(declaredScope, repo);
|
||||
const repoScope = resolveRepoDeclaredScope(declaredScope, repo, repoKeys).scope;
|
||||
const worktreesDir = path.resolve(resolveWorktreesDir(checkout, deps.settings, { workspaceRootDir: deps.rootDir, repoRelPath: repo }));
|
||||
const excluded = (file: string) => {
|
||||
const absolute = path.resolve(checkout, file);
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
* fn_task_done File Scope leak guard (workspace multi-repo + singular checkout).
|
||||
*/
|
||||
import type { Settings, Task, TaskStore } from "@fusion/core";
|
||||
import { deriveRepoScopeSubset } from "../worktree/workspace-paths.js";
|
||||
import { resolveRepoDeclaredScope } from "../worktree/workspace-paths.js";
|
||||
import { executorLog } from "../logger.js";
|
||||
import type { EngineRunContext, RunAuditor } from "../util/run-audit.js";
|
||||
import { parseReviewLevelFromPrompt } from "./prompt-derived-eligibility.js";
|
||||
@@ -114,7 +114,7 @@ export async function evaluateTaskDoneScopeLeak(
|
||||
const repoTouched = [...new Set([...repoUncommitted, ...repoCommitted])];
|
||||
// Repo-LOCAL declared-scope subset for THIS repo (prefix stripped). Same filter as the
|
||||
// non-workspace branch below — one surface.
|
||||
const repoScopeSubset = deriveRepoScopeSubset(declaredScope, repoRel);
|
||||
const repoScopeSubset = resolveRepoDeclaredScope(declaredScope, repoRel, repoKeys).scope;
|
||||
const repoOffScope = repoTouched
|
||||
.filter((filePath) => !workflowPathMatchesDeclaredScope(filePath, repoScopeSubset))
|
||||
.filter((filePath) => !isAlwaysAllowedScopeLeakPath(filePath))
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import type { Task, TaskStore } from "@fusion/core";
|
||||
import { deriveRepoForPath, deriveRepoScopeSubset, UNSCOPED_REPO } from "../worktree/workspace-paths.js";
|
||||
import { resolveRepoDeclaredScope } from "../worktree/workspace-paths.js";
|
||||
import { createCommitRangeFilesReader, enforceSquashFileScopeInvariant, FileScopeViolationError } from "./merger-file-scope.js";
|
||||
import type { RunAuditor } from "../util/run-audit.js";
|
||||
|
||||
@@ -10,13 +10,10 @@ const execFileAsync = promisify(execFile);
|
||||
export function resolveRepoDeclaredScopeTransform({ repoRel, repoKeys }: { repoRel: string; repoKeys: readonly string[] }) {
|
||||
return {
|
||||
transform(scope: string[]): string[] {
|
||||
const subset = deriveRepoScopeSubset(scope, repoRel);
|
||||
if (subset.length) return subset;
|
||||
return scope.some((entry) => deriveRepoForPath(entry, repoKeys) !== UNSCOPED_REPO) ? [] : scope;
|
||||
return resolveRepoDeclaredScope(scope, repoRel, repoKeys).scope;
|
||||
},
|
||||
describe(scope: string[]): "repo-subset" | "unprefixed-fallback" | "foreign-repo-only" {
|
||||
if (deriveRepoScopeSubset(scope, repoRel).length) return "repo-subset";
|
||||
return scope.some((entry) => deriveRepoForPath(entry, repoKeys) !== UNSCOPED_REPO) ? "foreign-repo-only" : "unprefixed-fallback";
|
||||
return resolveRepoDeclaredScope(scope, repoRel, repoKeys).source;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -901,7 +901,7 @@ export interface LandRepoContext {
|
||||
*/
|
||||
noCommitsExpected?: boolean;
|
||||
/** FNXC:Workspace 2026-08-15-08:36: Present only for workspace sub-repos; it fences the durable intent and remote ref advance. */
|
||||
workspaceLand?: { handle: WorkspaceLeaseHandle; repoRelPath: string; remote: string };
|
||||
workspaceLand?: { getHandle: () => WorkspaceLeaseHandle; repoRelPath: string; remote: string; assertLive: () => void };
|
||||
/** FNXC:WorkspaceMergeDispatch 2026-08-15-22:55: Task-level pin that fences every merge-body ref advance. */
|
||||
workspaceDispatchFence?: { fenceRefName: string; fenceRefSha: string };
|
||||
store: TaskStore;
|
||||
@@ -1168,7 +1168,9 @@ export async function landOneRepo(
|
||||
// move. A later reconciler can then settle an interrupted remote advance without re-squashing.
|
||||
let workspaceFence: { remote: string; fenceRefName: string; fenceRefSha: string } | undefined;
|
||||
if (ctx.workspaceLand) {
|
||||
const { handle, repoRelPath, remote } = ctx.workspaceLand;
|
||||
ctx.workspaceLand.assertLive();
|
||||
const { repoRelPath, remote } = ctx.workspaceLand;
|
||||
const handle = ctx.workspaceLand.getHandle();
|
||||
if (!handle.fenceRefName || !handle.fenceRefSha) {
|
||||
throw new Error(`Workspace land lease ${handle.leaseKey} is missing its fence pin`);
|
||||
}
|
||||
@@ -1186,6 +1188,7 @@ export async function landOneRepo(
|
||||
|
||||
// 4 + 5. Land the squash on the target branch and sync the user's
|
||||
// checkout (AI reconciles a conflicting restore).
|
||||
ctx.workspaceLand?.assertLive();
|
||||
await setStatus("landing");
|
||||
const landed = await landSquash({
|
||||
projectRootDir: repoRootDir, mergeRoot, integrationBranch, tipSha, squashSha, taskId, audit,
|
||||
@@ -2312,11 +2315,45 @@ export async function landWorkspaceTask(
|
||||
throw error;
|
||||
}
|
||||
|
||||
/*
|
||||
FNXC:Workspace 2026-08-20-19:45:
|
||||
A repository land can legitimately outlive the five-minute durable TTL while dependency sync and
|
||||
AI review run. Renew its current owner/fence handle during the complete land body, but keep every
|
||||
intent and push fenced by the durable handle: a failed renewal aborts before the next commit point.
|
||||
*/
|
||||
let leaseLost = false;
|
||||
const leaseAbort = new AbortController();
|
||||
let renewalInFlight: Promise<void> | undefined;
|
||||
const renewWorkspaceLease = (store as Partial<TaskStore>).renewWorkspaceLease;
|
||||
const renewLease = async (): Promise<void> => {
|
||||
if (!durableLandLease || typeof renewWorkspaceLease !== "function" || leaseLost) return;
|
||||
try {
|
||||
const renewed = await renewWorkspaceLease.call(store, durableLandLease, 5 * 60_000);
|
||||
if (!renewed) throw new Error("Workspace lease renewal was refused");
|
||||
durableLandLease = renewed;
|
||||
} catch {
|
||||
leaseLost = true;
|
||||
leaseAbort.abort("workspace-repo-land-lease-lost");
|
||||
}
|
||||
};
|
||||
const renewalTimer = durableLandLease && typeof renewWorkspaceLease === "function"
|
||||
? setInterval(() => {
|
||||
if (!renewalInFlight) {
|
||||
renewalInFlight = renewLease().finally(() => { renewalInFlight = undefined; });
|
||||
}
|
||||
}, 60_000)
|
||||
: undefined;
|
||||
renewalTimer?.unref?.();
|
||||
const landSignal = options.signal ? AbortSignal.any([options.signal, leaseAbort.signal]) : leaseAbort.signal;
|
||||
const assertLeaseLive = () => {
|
||||
if (leaseLost || leaseAbort.signal.aborted) throw new WorkspaceRepoLandBusyError(repoRel, "durable-workspace-lease", taskId);
|
||||
};
|
||||
|
||||
try {
|
||||
const landResult = await landOneRepo(repoRootDir, entry.branch, integrationBranch, {
|
||||
taskId, settings, audit, log, setStatus, maxPasses,
|
||||
mergeAgent, reviewAgent, stashResolveAgent,
|
||||
includeTaskId, trailers, taskTitle, signal: options.signal,
|
||||
includeTaskId, trailers, taskTitle, signal: landSignal,
|
||||
allowDirtyLocalCheckoutSync,
|
||||
// FNXC:Workspace 2026-06-24-23:50: one sub-repo's dependency-sync failure must not block
|
||||
// landing the others — degrade verification for that repo, still land the git squash.
|
||||
@@ -2325,10 +2362,11 @@ export async function landWorkspaceTask(
|
||||
noCommitsExpected: task.noCommitsExpected === true,
|
||||
repoRel,
|
||||
repoKeys,
|
||||
...(durableLandLease ? { workspaceLand: { handle: durableLandLease, repoRelPath: repoRel, remote: "origin" } } : {}),
|
||||
...(durableLandLease ? { workspaceLand: { getHandle: () => { assertLeaseLive(); return durableLandLease!; }, repoRelPath: repoRel, remote: "origin", assertLive: assertLeaseLive } } : {}),
|
||||
...(workspaceDispatchFence ? { workspaceDispatchFence } : {}),
|
||||
store,
|
||||
});
|
||||
assertLeaseLive();
|
||||
if (landResult.outcome === "landed") {
|
||||
/*
|
||||
FNXC:Workspace 2026-06-22-04:10 (Phase C review A1 — persist-after-advance is a HARD failure):
|
||||
@@ -2341,6 +2379,7 @@ export async function landWorkspaceTask(
|
||||
*/
|
||||
try {
|
||||
if (durableLandLease) {
|
||||
assertLeaseLive();
|
||||
const resolved = await store.resolveWorkspaceLandIntent({
|
||||
handle: durableLandLease,
|
||||
taskId,
|
||||
@@ -2379,6 +2418,12 @@ export async function landWorkspaceTask(
|
||||
repos.push({ repo: repoRel, repoRootDir, integrationBranch, branch: entry.branch, status: "empty" });
|
||||
}
|
||||
} catch (err: unknown) {
|
||||
/*
|
||||
FNXC:Workspace 2026-08-20-19:58:
|
||||
A lost repository tenancy is not a repository land failure: writing landFailure here would
|
||||
let a stale owner mutate workspace state after its durable lease was revoked.
|
||||
*/
|
||||
if (leaseLost) throw new WorkspaceRepoLandBusyError(repoRel, "durable-workspace-lease", taskId);
|
||||
if (isMergeAbortedError(err)) throw err;
|
||||
// A WorkspacePartialLandError from the persist-failure window above must PROPAGATE
|
||||
// (the engine parks/retries). The outer try/finally below resets status first (A3).
|
||||
@@ -2399,6 +2444,8 @@ export async function landWorkspaceTask(
|
||||
// loop and the landed predicate above skips them (only the failed repo retries).
|
||||
break;
|
||||
} finally {
|
||||
if (renewalTimer) clearInterval(renewalTimer);
|
||||
await renewalInFlight?.catch(() => undefined);
|
||||
/*
|
||||
FNXC:Workspace 2026-06-22-02:10 (Phase C U3, KTD4):
|
||||
Release the land lease — on land SUCCESS or land FAILURE — but ONLY when WE hold
|
||||
|
||||
@@ -113,3 +113,29 @@ export function deriveRepoScopeSubset(declaredScope: readonly string[], repoRel:
|
||||
}
|
||||
return subset;
|
||||
}
|
||||
|
||||
export type RepoDeclaredScopeResolution = {
|
||||
scope: string[];
|
||||
source: "repo-subset" | "unprefixed-fallback" | "foreign-repo-only";
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve File Scope exactly as one workspace repository sees it.
|
||||
*
|
||||
* FNXC:Workspace 2026-08-20-19:45:
|
||||
* Workspace merge, completion, and main-checkout guards must interpret task scope identically.
|
||||
* A matching repo prefix wins; only declarations with no configured repository owner may fall back
|
||||
* to repo-local patterns. A sibling-prefixed declaration is never authority for this repository.
|
||||
*/
|
||||
export function resolveRepoDeclaredScope(
|
||||
declaredScope: readonly string[],
|
||||
repoRel: string,
|
||||
repoKeys: readonly string[],
|
||||
): RepoDeclaredScopeResolution {
|
||||
const subset = deriveRepoScopeSubset(declaredScope, repoRel);
|
||||
if (subset.length > 0) return { scope: subset, source: "repo-subset" };
|
||||
const hasRepoOwnedEntry = declaredScope.some((entry) => deriveRepoForPath(entry, repoKeys) !== UNSCOPED_REPO);
|
||||
return hasRepoOwnedEntry
|
||||
? { scope: [], source: "foreign-repo-only" }
|
||||
: { scope: [...declaredScope], source: "unprefixed-fallback" };
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user