feat(KB-648): enable parallel test execution and optimize test performance

- Optimize backup tests using fake timers instead of real timeouts

- Enable parallel file execution in core, engine, CLI, and dashboard packages

- Add inline test helpers to reduce dependencies in dashboard routes tests

- Update executor tests with exact command matching and improved assertions

- Update AGENTS.md with test optimization patterns (fake timers, unique temp dirs)
This commit is contained in:
gsxdsm
2026-04-01 06:54:50 -07:00
parent e21ea42d45
commit fa37de1bed
79 changed files with 1568 additions and 7618 deletions

491
AGENTS.md
View File

@@ -41,7 +41,7 @@ kb uses a hybrid storage architecture: structured metadata lives in SQLite while
- **Project database:** `.kb/kb.db` — SQLite database with WAL mode enabled
- **Blob files:** `.kb/tasks/{ID}/PROMPT.md`, `agent.log`, `attachments/` — remain on filesystem
- **Global settings:** `~/.pi/fusion/settings.json` — remains file-based (not in SQLite)
- **Global settings:** `~/.pi/kb/settings.json` — remains file-based (not in SQLite)
### Tables
@@ -114,7 +114,7 @@ kb supports multi-project coordination through a central infrastructure that pro
### Central Database Location
The central database is stored at `~/.pi/fusion/fusion-central.db` (global user directory):
The central database is stored at `~/.pi/kb/kb-central.db` (global user directory):
| Table | Purpose |
|-------|---------|
@@ -210,7 +210,7 @@ kb has two activity log systems:
- Contains events for a single project
- Used by the dashboard for project-specific views
2. **Unified central activity log** (`~/.pi/fusion/fusion-central.db``centralActivityLog` table)
2. **Unified central activity log** (`~/.pi/kb/kb-central.db``centralActivityLog` table)
- Contains events from all projects
- Used for global dashboards and cross-project reporting
- Includes `projectId` and `projectName` for attribution
@@ -223,318 +223,6 @@ kb has two activity log systems:
- Cascade deletes ensure no orphaned data when projects are unregistered
- Foreign key constraints maintain referential integrity
## Multi-Project Runtime Architecture
kb's multi-project support is built on a runtime abstraction layer that enables task execution across multiple projects with configurable isolation modes. This architecture provides both efficiency (in-process) and security (child-process isolation) options.
### Overview
```
┌─────────────────────────────────────────────────────────────────┐
│ HybridExecutor │
│ (Multi-Project Orchestrator) │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ ProjectManager (internal) │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌─────────────────┐ │ │
│ │ │ Project A │ │ Project B │ │ Project C │ │ │
│ │ │ (in-process) │ │(child-process│ │ (in-process) │ │ │
│ │ └──────────────┘ └──────────────┘ └─────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
┌─────────┴──────────┐
▼ ▼
┌──────────┐ ┌──────────┐
│CentralCore│ │ Scheduler │
│ (registry)│ │ (per proj) │
└──────────┘ └──────────┘
```
### ProjectRuntime Interface
The `ProjectRuntime` interface is the core abstraction for multi-project execution:
```typescript
interface ProjectRuntime extends EventEmitter<ProjectRuntimeEvents> {
start(): Promise<void>; // Initialize and start the runtime
stop(): Promise<void>; // Graceful shutdown
getStatus(): RuntimeStatus; // Current runtime status
getTaskStore(): TaskStore; // Access project task store
getScheduler(): Scheduler; // Access project scheduler
getMetrics(): RuntimeMetrics; // Get current metrics
}
```
**RuntimeStatus values:** `starting``active``stopping``stopped`, or `paused`/`errored` for exceptional states.
### Runtime Implementations
#### 1. InProcessRuntime
Runs a project within the main Node.js process:
- **Pros:** Low overhead, fast startup (~0ms), shared memory, direct component access
- **Cons:** No isolation - crashes or resource leaks affect all projects
- **Use for:** Trusted projects, development, single-project deployments
```typescript
const runtime = new InProcessRuntime(config, centralCore);
await runtime.start();
// Direct access to components
const taskStore = runtime.getTaskStore();
const scheduler = runtime.getScheduler();
```
#### 2. ChildProcessRuntime
Runs a project in an isolated child process via `fork()`:
- **Pros:** Strong isolation, independent memory space, crash containment
- **Cons:** Higher overhead (~100-300ms startup), IPC communication overhead
- **Use for:** Untrusted projects, resource-intensive work, multi-tenant deployments
```typescript
const runtime = new ChildProcessRuntime(config, centralCore);
await runtime.start();
// Access via IPC only (getTaskStore/getScheduler throw)
const metrics = runtime.getMetrics();
```
### IPC Protocol
The IPC protocol enables communication between the host (HybridExecutor) and child process runtimes:
**Command Types (Host → Worker):**
- `START_RUNTIME` - Initialize and start the in-process runtime inside the child
- `STOP_RUNTIME` - Graceful shutdown with timeout
- `GET_STATUS` - Query runtime status
- `GET_METRICS` - Query runtime metrics
- `PING` - Health check
**Event Types (Worker → Host, unsolicited):**
- `TASK_CREATED` - Forwarded from TaskStore
- `TASK_MOVED` - Forwarded from TaskStore
- `TASK_UPDATED` - Forwarded from TaskStore
- `ERROR_EVENT` - Runtime errors
- `HEALTH_CHANGED` - Status transitions
### HybridExecutor
The `HybridExecutor` is the main entry point for multi-project task execution:
```typescript
const central = new CentralCore();
await central.init();
const executor = new HybridExecutor(central);
await executor.initialize();
// Add a project runtime
await executor.addProject({
projectId: "proj_abc123",
workingDirectory: "/path/to/project",
isolationMode: "in-process", // or "child-process"
maxConcurrent: 2,
maxWorktrees: 4,
});
// Listen for events across all projects
executor.on("task:completed", ({ projectId, taskId }) => {
console.log(`Task ${taskId} completed in ${projectId}`);
});
// Graceful shutdown
await executor.shutdown();
```
**Key Features:**
- Automatic loading of registered projects on initialization
- Event forwarding with project attribution
- Global concurrency limit enforcement via CentralCore
- Runtime mode switching (can change isolation mode)
- Health monitoring with automatic restart (child-process mode)
### Child Process Worker
The `child-process-worker.ts` entry point runs inside forked child processes:
1. Creates an `InProcessRuntime` internally
2. Sets up `IpcWorker` to handle commands from the host
3. Forwards all runtime events to the host via IPC
4. Handles graceful shutdown on SIGTERM
5. Self-terminates if parent disconnects unexpectedly
### Isolation Mode Selection
Choose isolation mode based on your requirements:
| Factor | In-Process | Child-Process |
|--------|-----------|---------------|
| Startup time | ~0ms | ~100-300ms |
| Memory isolation | No | Yes |
| Crash containment | No | Yes |
| IPC overhead | None | Minimal |
| Use case | Trusted/single | Multi-tenant/isolated |
### Security Considerations
- Child process spawn validates `projectPath` exists and is absolute before forking
- IPC message validation rejects malformed/unknown message types
- No credentials passed over IPC - credentials stay in parent
- Child process `cwd` is restricted to project path only
- Terminate child on parent exit (prevent orphaned processes)
- Input validation on all IPC message payloads
- Path traversal prevention in project path resolution
### Error Handling & Recovery
**Child Process Runtime:**
- Health monitoring via heartbeat every 5 seconds
- Automatic restart on crash with exponential backoff (1s, 5s, 15s delays)
- Max 3 restart attempts before transitioning to `errored` state
- Graceful shutdown timeout: 30 seconds (SIGTERM → SIGKILL after 5s)
**In-Process Runtime:**
- Errors are emitted as `error` events
- Status transitions to `errored` on fatal errors
- Manual intervention required to restart
## Multi-Project CLI Usage
The kb CLI supports managing multiple projects through the `kb project` subcommand and the `--project` global flag.
### Project Subcommands
```bash
# List all registered projects
kb project list
# Register a new project
kb project add my-app /path/to/app
# Unregister a project (data is preserved)
kb project remove my-app [--force]
# Show project details
kb project show my-app
# Set default project for CLI operations
kb project set-default my-app
# Detect which project you're currently in
kb project detect
```
### Global --project Flag
All task commands accept a `--project` (or `-P`) flag to target a specific project:
```bash
# Create a task in a specific project
kb task create "Fix login bug" --project my-app
# List tasks from a specific project
kb task list --project my-app
# Show task details from a project
kb task show KB-001 --project my-app
# Move a task to a different column
kb task move KB-001 done --project my-app
# Archive a completed task
kb task archive KB-001 --project my-app
# Delete a task
kb task delete KB-001 --force --project my-app
# Attach a file to a task
kb task attach KB-001 screenshot.png --project my-app
# Pause/unpause a task
kb task pause KB-001 --project my-app
kb task unpause KB-001 --project my-app
# Retry a failed task
kb task retry KB-001 --project my-app
# Create a PR for a task
kb task pr-create KB-001 --project my-app
# Import GitHub issues as tasks
kb task import owner/repo --project my-app
# Show and update settings for a project
kb settings --project my-app
kb settings set maxConcurrent 4 --project my-app
# Git operations in a project
kb git status --project my-app
kb git pull --project my-app
kb git push --project my-app
# Backup operations for a project
kb backup --create --project my-app
kb backup --list --project my-app
```
### Project Resolution Order
When you run a kb command without `--project`, the CLI resolves the project in this order:
1. **Explicit `--project` flag** — Uses the specified project
2. **Default project** — Uses the project set via `kb project set-default`
3. **CWD auto-detection** — Walks up the directory tree looking for `.fusion/kb.db`
If no project is found, the CLI exits with an error:
```
No kb project found in current directory. Use --project or run from a project directory.
```
### Common Workflows
**Cross-project operations without changing directories:**
```bash
# Create tasks in different projects from the same shell
kb task create "Backend API endpoint" --project api-service
kb task create "Frontend component" --project web-ui
kb task create "Documentation update" --project docs
# Check status of all projects
kb project list
# Archive completed tasks across projects
kb task archive API-042 --project api-service
kb task archive WEB-123 --project web-ui
```
**Setting up a default project:**
```bash
# Register your main project
kb project add main ~/projects/my-app
# Set it as default
kb project set-default main
# Now all commands use the default project without --project
kb task list
kb task create "New feature"
kb git status
```
**Switching between projects:**
```bash
# Quick switch with shell aliases
alias kb-api='kb --project api-service'
alias kb-web='kb --project web-ui'
# Or use the explicit flag
kb task list --project api-service
kb task list --project web-ui
```
## Pi Extension (`packages/cli/src/extension.ts`)
The pi extension provides tools and a `/kb` command for interacting with kb from within a pi session. It ships as part of `@gsxdsm/fusion` — one `pi install` gives you both the CLI and the extension.
@@ -616,14 +304,14 @@ Use `useBadgeWebSocket()` when a UI surface needs live badge snapshots for speci
kb uses a two-tier settings hierarchy:
- **Global settings** — User preferences stored in `~/.pi/fusion/settings.json`. These persist across all kb projects for the current user.
- **Global settings** — User preferences stored in `~/.pi/kb/settings.json`. These persist across all kb projects for the current user.
- **Project settings** — Project-specific workflow and resource settings stored in `.kb/config.json`. These control how the engine operates for a particular project.
When reading settings, project values override global values. The merged view is what the engine and dashboard use.
### Settings Hierarchy
**Global settings** (`~/.pi/fusion/settings.json`):
**Global settings** (`~/.pi/kb/settings.json`):
- `themeMode` — UI theme preference (dark/light/system)
- `colorTheme` — Color theme (default/ocean/forest/etc)
- `defaultProvider` — Default AI model provider
@@ -769,7 +457,7 @@ Controls how worktree directory names are generated when `recycleWorktrees` is N
**Valid values:**
- `"random"` — Human-friendly random names like `swift-falcon`, `calm-river` (default)
- `"task-id"` — Use the task ID as the directory name, e.g., `fn-042`
- `"task-id"` — Use the task ID as the directory name, e.g., `kb-042`
- `"task-title"` — Use a slugified version of the task title, e.g., `fix-login-bug`
**Example:**
@@ -783,7 +471,7 @@ Controls how worktree directory names are generated when `recycleWorktrees` is N
**Notes:**
- This setting has no effect when `recycleWorktrees` is enabled (pooled worktrees retain their existing names)
- Task branches are always named `fusion/{task-id}` regardless of this setting
- Task branches are always named `kb/{task-id}` regardless of this setting
- When using `"task-title"` mode, special characters are replaced with hyphens and the result is lowercased
### `autoBackupEnabled` (default: `false`)
@@ -1156,168 +844,3 @@ When you add a template:
1. The template data is copied to a new workflow step (templates themselves are immutable)
2. The new step is enabled by default
3. You can edit the step after creation to customize the prompt
## Multi-Project Migration
kb supports multi-project mode through a central infrastructure that coordinates across multiple kb projects. When upgrading from single-project to multi-project mode, the system provides automatic migration.
### Auto-Migration on First Run
When a user runs kb after the multi-project update, the system:
1. **Detects** existing `.kb/kb.db` files in the current directory and subdirectories (up to 5 levels deep)
2. **Auto-registers** valid kb projects in the central registry with `isolationMode: 'in-process'`
3. **Skips** projects that are already registered or lack valid databases
4. **Generates unique names** for projects (appending `-2`, `-3` for conflicts)
### Migration Behavior
```bash
# First run after update - auto-migration triggers automatically
fn task list
# [kb] First run detected. Auto-registering projects...
# [kb] Auto-registered 3 project(s):
# - my-app: /Users/me/projects/my-app
# - api-service: /Users/me/projects/api-service
# - docs: /Users/me/projects/docs
```
### Backward Compatibility
Single-project workflows continue to work seamlessly:
- **Existing constructor**: `new TaskStore(rootDir)` works unchanged
- **Auto-resolution**: CLI commands resolve the project from `--project` flag, CWD, or default project
- **Fallback mode**: If central database is unavailable, kb falls back to single-project mode
### Manual Migration
If auto-migration is skipped or fails, projects can be registered manually:
```bash
# Register current directory
fn project add .
# Register specific path
fn project add ~/projects/my-app
# List registered projects
fn project list
```
### First-Run Wizard
For new users (or fresh installs), the dashboard shows a first-run wizard when no projects are registered:
1. **Detection**: Scans for existing kb projects
2. **Selection**: User selects which projects to register
3. **Completion**: Projects are registered and ready for use
### Environment Variables
- `KB_SKIP_MIGRATION=1` — Disable auto-migration
- `FN_PROJECT=<name>` — Target specific project (set by `--project` flag)
### Recovery
If migration causes issues:
1. **Disable auto-migration**: Set `KB_SKIP_MIGRATION=1`
2. **Reset central DB**: Delete `~/.pi/kb/kb-central.db` (projects are preserved)
3. **Manual registration**: Use `fn project add <path>` after reset
## Multi-Project Dashboard
The kb dashboard supports managing multiple projects simultaneously. This enables teams to track tasks across multiple repositories from a single dashboard view.
### Overview
The multi-project dashboard provides:
- **Project Overview page** — Responsive grid showing all registered projects with health status
- **Project Selector** — Quick context switching between projects via header dropdown
- **Project Drill-down** — Click any project to view its tasks in board or list view
- **Setup Wizard** — First-run experience for registering projects
- **Global Activity Feed** — Cross-project activity with project attribution
### Project Status
Projects have one of these statuses:
- **`active`** — Project is operational and accepting tasks (green badge)
- **`paused`** — Project temporarily suspended (yellow badge)
- **`errored`** — Project has encountered errors (red badge)
- **`initializing`** — Project just registered, not fully set up (blue badge)
### Navigation
**View all projects:**
- Navigate to the Project Overview page showing all registered projects in a responsive grid (1→2→3 columns based on screen size)
- Filter by status: All, Active, Paused, Errored
- Sort by: Name, Last Activity, Status
**Switch projects:**
- Use the Project Selector in the header (visible when 2+ projects registered)
- Shows current project name with dropdown menu
- Displays project status icons for quick health assessment
- "View All Projects" option in dropdown returns to overview
**Back to overview:**
- "Back to All Projects" button appears in header when viewing a specific project
- Clicking returns to Project Overview page
### First-Run Experience
When no projects are registered:
1. Setup wizard automatically opens on dashboard load
2. Manually enter project path and name
3. Or auto-detect projects in a base directory
4. Select projects to register from detection results
5. Projects are initialized with `in-process` execution mode by default
### Project Health
Each project card shows:
- **Active Tasks** — Number of tasks in non-terminal columns
- **Agents** — Currently running agents for this project
- **Completed** — Total tasks completed (from health metrics)
- **Last activity** — Relative timestamp of last project activity
Health is polled every 10 seconds while Project Overview is visible.
### Activity Log
The global activity feed shows events from all projects:
- Project name badge on each entry
- Filter dropdown to show only specific project
- Filter by event type (same options as single-project view)
- Cross-project task linking (opens task detail if task exists in current view)
### Storage
Projects are registered in the central database (`~/.pi/fusion/fusion-central.db`):
- `projects` table — Project registry with path, status, isolation mode
- `projectHealth` — Mutable health metrics (active tasks, agent counts, totals)
- `centralActivityLog` — Unified activity feed across all projects
### View Preferences
View state is persisted per scope:
- **Overview vs Project mode** — `kb-dashboard-view-mode` in localStorage
- **Board/List/Agents view** — `kb-dashboard-task-view` in localStorage
- **Recently accessed projects** — Last 3 projects stored for quick selector access
### API Integration
Multi-project components use these APIs:
- `GET /api/projects` — List all registered projects
- `POST /api/projects` — Register new project
- `PATCH /api/projects/:id` — Update project status/name
- `DELETE /api/projects/:id` — Unregister project
- `GET /api/projects/:id/health` — Fetch project health metrics
- `GET /api/activity-feed` — Global activity (supports `?projectId=` filter)
### Backward Compatibility
Single-project mode is automatically maintained:
- With only 1 project, Project Selector is hidden
- Dashboard behaves like existing single-project mode
- View preference falls back to "project" mode automatically