root/fusion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

FN-5902: make mission validation AI-run all criteria

Browse Source 
Route every mission feature through validator-backed completion checks.

- lazily restore a managed feature assertion before validation instead of auto-passing zero-assertion features
- thread milestone acceptance criteria into validator prompts and system instructions as enforced requirements
- update MissionManager copy/tests to present criteria as AI-validated runtime gates and remove informational-only/zero-assertion warnings
- document the all-criteria AI-run contract and add a changeset for @runfusion/fusion

Files changed:
 .changeset/fn-5902-mission-validation-ai-run.md    |   5 +
 AGENTS.md                                          |   2 +-
 docs/architecture.md                               |   2 +-
 docs/missions-completion-contract.md               | 198 ++++++---------------
 docs/missions.md                                   |   5 +-
 packages/core/src/__tests__/mission-store.test.ts  |  23 ++-
 packages/core/src/mission-store.ts                 |  10 ++
 packages/dashboard/app/components/MissionManager.css    |  31 ----
 packages/dashboard/app/components/MissionManager.tsx    |  86 +++------
 packages/dashboard/app/components/__tests__/MissionManager.test.tsx   |  60 +++++--
 packages/engine/src/__tests__/mission-execution-loop.test.ts   | 111 +++++++++---
 packages/engine/src/__tests__/reliability-interactions/mission-validation-trigger-gap.test.ts         |  57 +++---
 packages/engine/src/mission-execution-loop.ts      |  78 ++++----
 13 files changed, 318 insertions(+), 350 deletions(-)

Fusion-Task-Id: FN-5902

Fusion-Task-Lineage: 5f25caad-33c9-42ff-822b-1ea092afc29f
This commit is contained in:
gsxdsm
2026-06-02 18:11:26 -07:00
parent abbeaec0a8
commit cc18206bc5
13 changed files with 318 additions and 350 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/architecture.md
View File

@@ -1779,7 +1779,7 @@ Reliability-layer changes are in scope. Interaction regression backstops live in
- FN-5830 backstop: `packages/engine/src/__tests__/reliability-interactions/branch-group-promotion.test.ts` guards branch-group completion-gate + promotion lifecycle so promotion happens exactly once after all members land, re-calls are idempotent, and gated paths emit `merge:branch-group-promotion-gated` without default-branch promotion.
- FN-5819/FN-5846 backstop: `packages/engine/src/__tests__/reliability-interactions/shared-group-member-integration.test.ts` and `shared-branch-group-lifecycle.test.ts` guard the scoped autoMerge-off exception and deterministic finalize path so shared members integrate into the single group branch, produce `mergeTargetSource: "branch-group-integration"`/`mergeTargetBranch`, do not land on main, and are not moved backward by self-healing maintenance.
- FN-5901 backstop: `packages/engine/src/__tests__/reliability-interactions/mission-validator-run-reaper.test.ts` guards stale mission-validator-run recovery across manual and automatic trigger types, verifies `mission:validator-run-reaped` audit metadata, ensures archived/complete parents keep their terminal feature state untouched, and proves reaped active features resume validation instead of staying wedged behind abandoned `running` rows.
- FN-5738 backstop: `packages/engine/src/__tests__/reliability-interactions/mission-validation-trigger-gap.test.ts` extends coverage so zero-assertion auto-pass deterministically advances `loopState` to `passed`, sets `lastValidatorStatus="passed"`, emits `validation_auto_passed_no_assertions`, and does not re-fire on repeated recovery passes.
- FN-5738 backstop (superseded by FN-5902): `packages/engine/src/__tests__/reliability-interactions/mission-validation-trigger-gap.test.ts` no longer permits zero-assertion auto-pass. Current coverage proves legacy zero-link features lazily restore a managed assertion, route through validator runs, and do not emit `validation_auto_passed_no_assertions` during recovery replays.
- FN-5741 backstop: `packages/engine/src/__tests__/reliability-interactions/merge-request-shadow-handoff.test.ts` guards Phase-1 merge-request contract shadow writes: flag OFF is a no-op, flag ON writes marker/record strictly after legacy handoff, and `autoMerge:false` remains `manual-required` without shadow running transitions.
- FN-5742 backstop: `packages/engine/src/__tests__/reliability-interactions/dual-observe-merge-seam.test.ts` guards Phase-2 dual-observe invariants: legacy dependency satisfaction remains authoritative while parity diffs emit, and shadow dequeue selection never advances `manual-required` rows.
- FN-5743 backstop: `packages/engine/src/__tests__/reliability-interactions/merge-request-cancel-on-hard-cancel.test.ts` and `packages/core/src/__tests__/merge-request-record.test.ts` guard Phase-3 cutover invariants: transient merge retries mutate merge-request state (no column rebound), user hard-cancel after accepted handoff cancels pending merge requests, and non-user rebounds preserve legacy fail-soft semantics.

198
docs/missions-completion-contract.md
View File

@@ -2,175 +2,93 @@
## Status
- **Decision date:** 2026-05-30
- **Task:** FN-5718
- **Depends on enforcement behavior from:** FN-5715 (reference implementation of the trigger/recovery path)
- **Scope:** Product contract and implementation requirements only (no code changes in this task)
- **Implementation status:** Realized by FN-5733 (loop auto-pass advancement, mission/store guard telemetry, MissionManager label reconciliation)
- **Decision date:** 2026-06-02
- **Current contract task:** FN-5902
- **Supersedes:** FN-5718 baseline contract
- **Depends on runtime trigger/recovery behavior from:** FN-5715
- **Implementation status:** Realized by FN-5902
## Problem
## Decision
Mission validation has had a recurring ambiguity: users can see feature acceptance text and milestone "completion criteria" text, but autopilot enforcement is actually driven by assertion linkage + validator outcomes. This document defines the canonical enforced gate so mission completion cannot silently stall or be misread.
Mission completion now uses an **all-criteria AI-run contract**:
## Canonical Completion Gate (Enforced)
### Decision
A feature is autopilot-complete only when **its linked contract assertions are satisfied**.
Canonical authored source and enforcement path:
1. `MissionFeature.acceptanceCriteria` is the canonical authored criteria text (authoring surface).
2. MissionStore must maintain a **store-managed per-feature `MissionContractAssertion`** derived from feature content, with text priority:
1. `MissionFeature.acceptanceCriteria` is the canonical authored feature criteria text.
2. MissionStore must maintain or lazily restore **one store-managed per-feature `MissionContractAssertion`** derived from feature content with text priority:
- `feature.acceptanceCriteria`
- `feature.description`
- `Verify implementation of: {feature.title}`
3. The mission validator enforces completion using the feature's **linked assertions** (including its store-managed assertion and any additional linked milestone assertions).
4. Feature/slice/mission advance is gated by the validator outcome (or explicit no-assertions auto-pass behavior defined below).
3. The mission validator must run for every feature completion trigger. Runtime validation may lazily call `ensureFeatureAssertionLinked(feature.id)` before starting the validator so legacy missing-link rows still become validator-backed.
4. `milestone.acceptanceCriteria` is also part of the enforced gate by being threaded into the validator prompt for every feature in that milestone.
5. Feature, slice, milestone, and mission advancement are gated by the validator result — **not** by an informational-only path.
### Precedence and interpretation rules
## Enforcement Model
- `MissionFeature.acceptanceCriteria` is the canonical authoring field for feature-level intent.
- The **linked assertion set** is the canonical enforcement set.
- Milestone `MissionContractAssertion` rows are additive contract rows. They are enforced **only when linked to a feature**.
- `milestone.acceptanceCriteria` is descriptive/informational milestone text and is not directly executed by the validator.
### Feature-level enforcement
### Worked examples
A feature is autopilot-complete only when the validator passes after evaluating:
1. **Feature has acceptance criteria; store-managed assertion linked; validator passes**
- Result: feature may move to done/passed and contribute to slice completion.
2. **Feature has acceptance criteria; additive milestone assertion also linked; one linked assertion fails**
- Result: feature is not complete; no slice advance.
3. **Feature has acceptance criteria visible, but no linked assertions (legacy FN-5696 shape)**
- Result: data inconsistency; must not be interpreted by operators as a separate enforced gate. Repair links (FN-5696 backfill) so enforcement matches displayed intent.
- the feature's linked contract assertions, including its store-managed assertion, and
- the parent milestone's `acceptanceCriteria` text when present.
## Enforced vs. Informational Surfaces
### Milestone-level enforcement
| Surface | Category | Contract meaning |
|---|---|---|
| `MissionFeature.acceptanceCriteria` | Informational authoring source | Canonical authored feature criteria text; enforcement happens through derived/linked assertions |
| Store-managed per-feature `MissionContractAssertion` | Enforced | Primary validator gate for the feature |
| Additive milestone `MissionContractAssertion` (linked to feature) | Enforced | Additional validator gate for that linked feature |
| Additive milestone `MissionContractAssertion` (unlinked) | Informational until linked | Contract candidate, not yet a feature gate |
| `milestone.acceptanceCriteria` | Informational | Milestone summary/pass-bar text for humans; not directly validator-executed |
| MissionManager `milestone-feature-acceptance-rollup` UI (`data-testid="milestone-feature-acceptance-rollup"`) | Informational display | Display-only rendering of feature acceptance text, not a separate enforcement mechanism |
`milestone.acceptanceCriteria` is no longer informational-only. FN-5902 enforces it by threading the milestone pass-bar text into the validator prompt for each feature under that milestone.
## Zero-Assertion Behavior and FN-5696 Failure Shape
This is intentionally **prompt-threading**, not per-feature milestone assertion row synthesis:
### Zero-assertions runtime behavior (canonical FN-5738 path)
- store-managed per-feature assertions remain the canonical feature assertion rows,
- milestone acceptance text remains milestone-authored prose,
- the validator sees both and must satisfy both.
When a feature reaches completion trigger points and has **zero linked assertions**, mission execution must take exactly one canonical auto-pass path (not a silent stall and not a competing behavior):
### Legacy data and lazy repair
- mark feature terminal as `status="done"`, `loopState="passed"`, `lastValidatorStatus="passed"`,
- emit explicit observability/audit evidence with mission event code `validation_auto_passed_no_assertions`,
- continue normal slice/mission advancement checks idempotently (no duplicate re-fire on repeated recovery).
Legacy missions can still contain features with missing assertion links. Runtime enforcement no longer depends on pre-running backfill:
### FN-5696 legacy shape clarification
- mission execution lazily restores the store-managed feature assertion just before validation, and
- `fn_mission_backfill_assertions` / `backfillFeatureAssertions()` remain available as operator repair tooling for data hygiene and visibility.
A feature can show acceptance text while links are missing (legacy pre-repair data). This must be treated as a **linkage/data integrity problem**, not as proof that milestone text alone is enforced. Assertion authoring/backfill (FN-5696) is outside the execution loop; the loop must not synthesize `mission_feature_assertions` rows. The contract prevents ambiguity by separating:
## Removed behavior (FN-5902 inversion)
- authored/informational text surfaces, from
- linked assertion enforcement surfaces.
FN-5718's zero-assertion auto-pass behavior is superseded.
Operators should use the mission assertion backfill operator path to restore expected store-managed linkage for FN-5696 legacy rows:
Removed contract:
- Agent/tool: `fn_mission_backfill_assertions` with `{ missionId?, dryRun? }` (defaults to dry-run).
- API: `POST /api/missions/:missionId/backfill-assertions` with body `{ dryRun?: boolean }` (defaults to `true`).
- Run dry-run first, then apply (`dryRun=false`) once repaired rows look correct.
- This remediation is additive: it derives/links one store-managed assertion per unlinked feature so runtime enforcement uses validator-linked assertions rather than the zero-assertion auto-pass branch.
- no `validation_auto_passed_no_assertions` completion path,
- no silent or explicit rubber-stamp pass because assertions were missing,
- no informational-only feature criteria bucket in MissionManager.
## Slice Status and Mission Autopilot Advance Derivation
Instead, features are routed through validator execution after lazy assertion ensure.
Autopilot may advance only when each active-slice feature is resolved under this contract:
## Worked examples
- Feature with linked assertions: all linked assertions must pass.
- Feature with zero linked assertions: explicit auto-pass path completes it.
- Feature with failed/blocked validation: slice remains incomplete.
- Feature stranded without a task link in an active autopilot slice (`taskId == null`): startup + maintenance reconciliation must repair it (title-match link first, otherwise defined-status re-triage) so `allDone` remains reachable instead of stalling on never-triaged features.
1. **Feature has acceptance criteria; no linked assertion row is present yet**
- Runtime calls `ensureFeatureAssertionLinked(feature.id)`.
- Validator runs against the restored managed assertion.
- Result gates completion normally.
Then:
2. **Feature has acceptance criteria and milestone acceptance criteria**
- Validator evaluates the linked feature assertion(s).
- Validator also evaluates the milestone acceptance text in the prompt.
- Feature passes only when both are satisfied.
1. All features resolved complete → slice flips to `complete`.
2. Completed active slice with pending next slice → next slice activates.
3. All milestone slices complete → milestone complete.
4. All mission milestones complete → mission complete.
3. **Operator runs backfill on legacy data**
- Backfill pre-restores missing managed assertions for visibility/reporting.
- Runtime behavior is unchanged because lazy ensure already guarantees validator-backed enforcement.
This keeps completion logic deterministic and consistent with FN-5715 trigger/recovery behavior.
## UI contract
## UI Reconciliation Requirements (for follow-on engineering task)
MissionManager must present mission criteria as **AI-validated** rather than informational:
✅ Implemented in FN-5733 with MissionManager labels:
- `Contract assertions (autopilot gate)` + enforced indicator
- `Feature acceptance criteria (informational)` + not-enforced indicator
- warning badge when `hasProseButNoAssertions === true`
- assertion heading text reflects AI validation,
- informational / not-enforced labels are removed,
- zero-assertion warning guard is removed,
- fallback feature-criteria rollups, when shown for missing loaded assertions, describe runtime AI validation rather than non-enforced prose.
Target surface: `packages/dashboard/app/components/MissionManager.tsx`
## Success invariant
1. **Disambiguate labels**
- Use distinct wording for:
- feature-authored acceptance text, and
- milestone contract assertions.
- Do not reuse "Completion criteria" to refer to both categories.
- Required wording baseline (or semantically equivalent copy):
- Feature rollup heading: `Feature acceptance criteria (informational source)`
- Assertion list heading: `Contract assertions (validator-enforced when linked)`
For any mission feature that reaches validation trigger points:
2. **Per-row enforcement indicator**
- Every displayed row in the assertions/criteria area must show whether it is:
- `Enforced gate` (validator-blocking when linked), or
- `Informational` (display-only).
3. **Empty-state contract-correct copy**
- Replace the current implication that completion criteria are absent when assertion rows are empty.
- Empty-state text must acknowledge when feature acceptance text exists but no assertion rows are defined/linked.
- Required behavior:
- If feature acceptance text exists but no assertion rows are present, show copy equivalent to: `No contract assertions are linked yet. Feature acceptance criteria are present below and remain informational until assertions are linked.`
- If neither feature acceptance text nor assertions exist, show copy equivalent to: `No feature acceptance criteria or contract assertions defined yet.`
4. **No button/mobile scope expansion**
- No button touch-target/mobile-reflow requirements (standing directive).
## Engineering Acceptance Criteria (follow-on implementation)
✅ Implemented in FN-5733:
- Auto-pass path now advances `loopState` to `passed` and emits mission event code `validation_auto_passed_no_assertions` while preserving the `validation:passed` emit contract (`"No assertions linked"` summary).
- Milestone rollup/store guard now exposes `hasProseButNoAssertions` and emits warning mission event code `milestone_missing_structured_assertions` (debounced on transition into condition).
- MissionManager UI now distinguishes enforced assertion gate vs informational feature acceptance criteria.
1. **Data/model contract**
- Preserve the canonical relationship: feature-authored criteria -> store-managed assertion -> linked assertion enforcement.
- If any model/UI metadata is added for enforced-vs-informational badges, it must be backward compatible with existing mission rows.
2. **Validator/loop behavior**
- Maintain FN-5715 invariants:
- done mission-linked tasks with linked assertions trigger validation,
- completion-trigger starts loop if needed,
- startup recovery replays done-implementing features with unpassed assertions,
- periodic self-heal maintenance replays the same `recoverActiveMissions` path so historically stranded `implementing` features recover without restart,
- zero-linked-assertions path remains explicit canonical auto-pass.
3. **UI behavior**
- Implement the Step-2 label reconciliation and per-row indicator requirements.
- Ensure no shared ambiguous terminology remains between feature acceptance text and assertion rows.
4. **Regression coverage**
- Add at least one regression test pinning the Goals-mission shape:
- feature has `acceptanceCriteria`,
- parent milestone has zero `MissionContractAssertion` rows / no links,
- autopilot behavior is deterministic and observable (explicit auto-pass path, no silent stall).
5. **Operational observability**
- Ensure mission/audit surfaces make no-assertions auto-pass and subsequent advance decisions queryable in logs/events.
## Success Metric
For 30 days after the follow-on implementation ships:
- **Primary metric:** zero autopilot stalls of the FN-5715 class (done mission task + unresolved validation trigger gap) in production mission runs.
- **Evidence source:** mission audit/event stream (`feature_completed`, `slice_completed`, `mission_completed`) plus `mission_validator_runs` records showing:
- explicit no-assertions auto-pass evidence (summary/reason path such as `No assertions linked` when no validator run is started), and
- downstream advancement evidence without stalled active slices.
## Follow-on Task Requirement
Implementation must land in a separate engineering task that references this document and FN-5715 as the enforcement baseline.
- a validator run must occur,
- the feature must not auto-pass due to missing assertion links,
- milestone acceptance text must be visible to the validator when present,
- advancement decisions must derive from validator outcomes only.

5
docs/missions.md
View File

@@ -512,7 +512,7 @@ A feature transitions to `blocked` when:
On engine restart, `recoverActiveMissions()` re-enqueues features in `validating` or `needs_fix` states, ensuring no validation work is lost. It also re-triggers `implementing` features whose linked task is already `done`/`archived` and whose assertion validation has not passed yet. When the stale-run reaper has already converted an abandoned validator run into `needs_fix`, `processTaskOutcome()` promotes the feature back through `implementing` and re-validates instead of skipping it. The same recovery path is replayed during periodic self-heal maintenance, so historically stranded `implementing` features can self-heal without requiring an engine restart.
For features with zero linked assertions, the completion path is explicit: the loop marks the feature `done`, advances `loopState` to `passed`, emits `validation:passed` with summary `"No assertions linked"`, and records mission event code `validation_auto_passed_no_assertions`. Contract details (including canonical no-assertions behavior and FN-5696 assertion-authoring separation) are defined in [Mission Completion Gate Contract](./missions-completion-contract.md).
For features with missing linked assertions, the completion path is now validator-first: the loop lazily restores the store-managed per-feature assertion just before validation, then runs the AI validator instead of auto-passing. Milestone `acceptanceCriteria` is threaded into the validator prompt for every feature in that milestone, so all mission criteria are AI-evaluated. Contract details are defined in [Mission Completion Gate Contract](./missions-completion-contract.md).
### Autopilot / Scheduler Interplay
@@ -543,8 +543,7 @@ These are independent tracking mechanisms — autopilot monitors mission progres
**MissionEvent audit types:**
- `slice_activated`, `feature_planned`, `feature_completed`
- `validation:started`, `validation:passed`, `validation:failed`, `validation:blocked`
- `validation_auto_passed_no_assertions` (reason: `"No assertions linked"`)
- `milestone_missing_structured_assertions` (warning when prose criteria exist with zero structured assertions)
- `milestone_missing_structured_assertions` (legacy-data warning surface; enforcement still lazy-restores managed assertions at runtime)
- `fix_feature:created`, `feature:blocked`
**Validator run telemetry:**