test(integration): user testing validation round 1 — 5/12 assertions passed

6 flow validators tested all integration milestone assertions against
the live dashboard and API. 5 assertions passed (E2E validation cycle,
fix cycle, loop state visuals, validate button, responsive). 7 failed
due to 3 systemic issues: (1) assertions panel renders empty despite
correct API data, (2) run history TypeError crash (.map on undefined),
(3) missing UI components (rollup badge, retry budget, lineage link).

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
gsxdsm
2026-04-12 09:02:57 -07:00
parent 274600322f
commit 8872fadfb3
8 changed files with 929 additions and 0 deletions

View File

@@ -26,3 +26,73 @@ Testing surface, required testing skills/tools, and resource cost classification
- Dashboard dev server (`fn dashboard`) needs to be running for browser tests
- API tests via curl are very lightweight — no concurrency limit needed
- Tests should use a fresh `.fusion/fusion.db` to avoid state pollution
## Test Data Context
**Mission:** M-MNVT98HS-I8OG ("Integration Test Mission")
**Milestone:** MS-MNVT9VEC-70GM ("Validation Test Milestone")
**Slices:** SL-MNVTAC2B-49SK (Slice 1), SL-MNVTAC91-98T3 (Slice 2)
**Assertions:**
- CA-MNVTGDE4-YEGX: "Feature links correctly" (pending)
- CA-MNVTGDHD-8ZAJ: "Validation passes on success" (pending)
- CA-MNVTGDRD-QNBH: "Fix feature created on failure" (pending)
**Features with loop states:**
| ID | Title | Loop State | Status | Notes |
|----|-------|-----------|--------|-------|
| F-MNVTCGT6-Z6PM | Assertion Linking Test | idle | defined | First feature, no links |
| F-MNVTDFNW-NXSW | Assertion Linking Feature | idle | triaged | Linked to CA-...YEGX, CA-...8ZAJ |
| F-MNVTDFQ1-ED3P | Implementing State Feature | implementing | in-progress | No assertions |
| F-MNVTDG3I-YZCR | Validating State Feature | validating | in-progress | No assertions |
| F-MNVTDGC0-J3W3 | Needs Fix Feature | needs_fix | in-progress | Linked to CA-...QNBH, 1 failed run |
| F-MNVTDGJ7-N7QC | Passed Validation Feature | passed | done | Linked to CA-...YEGX, CA-...8ZAJ, 1 passed run |
| F-MNVTDGLC-G6TD | Blocked Feature | blocked | in-progress | No assertions, budget exhausted |
| F-MNVTDGXO-EU7E | Fix Feature from Lineage | passed | done | generatedFrom F-...J3W3, has lineage |
| F-MNVTDH2E-YYZD | Run History Feature | passed | done | 3 runs (2 failed, 1 passed) |
**Dashboard URL:** http://localhost:4040
**Navigation:** Click "Missions" in sidebar → find "Integration Test Mission" → expand
## Flow Validator Guidance: Dashboard UI
**Isolation:** All browser validators share the same dashboard and test data. No mutations needed — validators observe existing state. Concurrent execution is safe.
**Navigation pattern:**
1. Go to http://localhost:4040
2. Click "Missions" tab in sidebar navigation
3. Find "Integration Test Mission" and click to expand
4. The mission detail shows milestones, slices, features, and assertions
**Key selectors and patterns:**
- Mission list items: `.mission-item` or similar
- Milestone expand/collapse: click milestone header
- Feature cards: look for feature titles and loop state indicators
- Assertions panel: within milestone detail, look for assertions section
- Validation rollup: milestone header area
**Constraints:**
- Do NOT create/delete any data through the browser — only observe
- Do NOT modify the test mission data
- Each validator should take screenshots as evidence
- Use `--session` flag for all agent-browser calls to avoid session conflicts
## Flow Validator Guidance: REST API
**Isolation:** API tests can run freely alongside browser tests. No shared browser session.
**Base URL:** http://localhost:4040
**Key endpoints:**
- `GET /api/missions` — list all missions
- `GET /api/missions/:id` — mission detail with milestones/slices/features
- `GET /api/missions/milestones/:id/assertions` — assertions for milestone
- `GET /api/missions/features/:id/assertions` — linked assertions for feature
- `GET /api/missions/features/:id/validation-loop` — loop state snapshot
- `GET /api/missions/features/:id/validation-runs` — validator run history
- `GET /api/missions/milestones/:id/validation` — milestone rollup
- `GET /api/missions/validation-runs/:id` — single run detail
**Constraints:**
- Read-only testing preferred
- If creating test data, clean up afterward