feat(FN-3706): gate research tools behind experimental feature

The merge introduces a research tool surface gating mechanism: shared availability helpers in core and engine, applied to the executor and triage agent so research prompts and tool exposure are gated behind experimental-feature flags, with tests covering the new logic.

Fusion-Task-Id: FN-3706
This commit is contained in:
Fusion
2026-05-07 15:50:20 -07:00
committed by gsxdsm
parent 8bbb7346b8
commit c38b7cd002
11 changed files with 231 additions and 24 deletions

View File

@@ -0,0 +1,5 @@
---
"@runfusion/fusion": patch
---
Gate research tool exposure in planning and execution sessions behind `experimentalFeatures.researchView`, including conditional prompt guidance so agents only see `fn_research_*` references when those tools are actually registered.

View File

@@ -291,7 +291,7 @@ This layered behavior is shared by heartbeat agents and task-scoped sessions tha
## Research Tools in Planning/Execution Sessions ## Research Tools in Planning/Execution Sessions
Triage and executor runtime sessions now include a bounded research tool surface: Triage and executor runtime sessions include a bounded research tool surface only when `experimentalFeatures.researchView` is enabled for the project:
- `fn_research_run` — create/start a bounded research run for a focused query - `fn_research_run` — create/start a bounded research run for a focused query
- `fn_research_list` — list recent runs and statuses - `fn_research_list` — list recent runs and statuses
@@ -304,7 +304,8 @@ Expected behavior and boundaries:
- Agents should use research only when repository/local context is insufficient - Agents should use research only when repository/local context is insufficient
- Queries should stay narrow and task-scoped; avoid open-ended exploration - Queries should stay narrow and task-scoped; avoid open-ended exploration
- If research is disabled or provider setup is incomplete, tools return actionable `setup` responses instead of crashing - When `experimentalFeatures.researchView` is disabled, sessions do not register `fn_research_*` tools and prompts do not advertise research capabilities
- If the research surface is enabled but provider setup is incomplete, tools return actionable `setup` responses instead of crashing
- Durable conclusions should be persisted with `fn_task_document_write` (for example, `key="research"`) - Durable conclusions should be persisted with `fn_task_document_write` (for example, `key="research"`)
- Research runs require the project engine to be running for processing; `fn_research_run` creates the run but does not block for completion unless `wait_for_completion` is set - Research runs require the project engine to be running for processing; `fn_research_run` creates the run but does not block for completion unless `wait_for_completion` is set

View File

@@ -0,0 +1,24 @@
import type { Settings } from "./types.js";
const LEGACY_EXPERIMENTAL_FEATURE_ALIASES: Record<string, string> = {
devServer: "devServerView",
};
export function isExperimentalFeatureEnabled(
settings: Pick<Settings, "experimentalFeatures"> | undefined,
key: string,
): boolean {
const features = settings?.experimentalFeatures;
if (!features) return false;
const canonicalKey = LEGACY_EXPERIMENTAL_FEATURE_ALIASES[key] ?? key;
if (features[canonicalKey] === true) return true;
for (const [legacyKey, aliasCanonical] of Object.entries(LEGACY_EXPERIMENTAL_FEATURE_ALIASES)) {
if (aliasCanonical === canonicalKey && features[legacyKey] === true) {
return true;
}
}
return false;
}

View File

@@ -740,6 +740,7 @@ export type {
ResearchCancellationState, ResearchCancellationState,
} from "./research-types.js"; } from "./research-types.js";
export { isExperimentalFeatureEnabled } from "./experimental-features.js";
export { isResearchExperimentalEnabled, resolveResearchSettings } from "./research-settings.js"; export { isResearchExperimentalEnabled, resolveResearchSettings } from "./research-settings.js";
export type { ResolvedResearchSettings } from "./research-settings.js"; export type { ResolvedResearchSettings } from "./research-settings.js";
export { resolveEvalSettings } from "./eval-settings.js"; export { resolveEvalSettings } from "./eval-settings.js";

View File

@@ -1,7 +1,8 @@
import type { ResearchEnabledSources, Settings } from "./types.js"; import type { ResearchEnabledSources, Settings } from "./types.js";
import { isExperimentalFeatureEnabled } from "./experimental-features.js";
export function isResearchExperimentalEnabled(settings: Partial<Settings> | undefined): boolean { export function isResearchExperimentalEnabled(settings: Partial<Settings> | undefined): boolean {
return settings?.experimentalFeatures?.researchView === true; return isExperimentalFeatureEnabled(settings, "researchView");
} }
export interface ResolvedResearchSettings { export interface ResolvedResearchSettings {

View File

@@ -19,6 +19,7 @@ import {
} from "../agent-tools.js"; } from "../agent-tools.js";
import * as core from "@fusion/core"; import * as core from "@fusion/core";
import type { MessageStore, Message } from "@fusion/core"; import type { MessageStore, Message } from "@fusion/core";
import { getEnabledPluginTools, getResearchToolSurfaceStatus } from "../tool-availability.js";
const loggerSpies = vi.hoisted(() => ({ const loggerSpies = vi.hoisted(() => ({
log: vi.fn(), log: vi.fn(),
@@ -60,6 +61,30 @@ vi.mock("node:child_process", async () => {
}; };
}); });
describe("tool availability helpers", () => {
it("treats research surface as disabled when researchView experimental flag is off", () => {
expect(getResearchToolSurfaceStatus({ experimentalFeatures: { researchView: false } } as any)).toEqual({
enabled: false,
reason: "experimental-disabled",
});
});
it("treats research surface as enabled when researchView experimental flag is on", () => {
expect(getResearchToolSurfaceStatus({ experimentalFeatures: { researchView: true } } as any)).toEqual({
enabled: true,
reason: "enabled",
});
});
it("resolves legacy experimental feature aliases through core helper", () => {
expect(core.isExperimentalFeatureEnabled({ experimentalFeatures: { devServer: true } } as any, "devServerView")).toBe(true);
});
it("returns no plugin tools when plugin runner is absent", () => {
expect(getEnabledPluginTools(undefined)).toEqual([]);
});
});
describe("createTaskCreateTool", () => { describe("createTaskCreateTool", () => {
it("returns details.taskId and keeps Created <id> response text", async () => { it("returns details.taskId and keeps Created <id> response text", async () => {
const store = { const store = {

View File

@@ -4938,6 +4938,72 @@ describe("Code review verdict enforcement - fn_task_update blocking", () => {
expect(result.content[0].text).toContain('type="code"'); expect(result.content[0].text).toContain('type="code"');
}); });
it("omits research prompt guidance when researchView experimental flag is disabled", async () => {
let capturedSystemPrompt = "";
mockedCreateFnAgent.mockImplementation(async (opts: any) => {
capturedSystemPrompt = opts.systemPrompt || "";
return {
session: {
prompt: vi.fn().mockResolvedValue(undefined),
dispose: vi.fn(),
sessionManager: { getLeafId: vi.fn(), branchWithSummary: vi.fn() },
navigateTree: vi.fn(),
},
} as any;
});
const store = createMockStore();
store.getSettings.mockResolvedValue({ ...(await store.getSettings()), experimentalFeatures: { researchView: false } });
const executor = new TaskExecutor(store, "/tmp/test");
await executor.execute({
id: "FN-SYS-NO-RESEARCH",
title: "Test",
description: "Test",
column: "in-progress",
dependencies: [],
steps: [],
currentStep: 0,
log: [],
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
});
expect(capturedSystemPrompt).not.toContain("fn_research_run");
});
it("includes research prompt guidance when researchView experimental flag is enabled", async () => {
let capturedSystemPrompt = "";
mockedCreateFnAgent.mockImplementation(async (opts: any) => {
capturedSystemPrompt = opts.systemPrompt || "";
return {
session: {
prompt: vi.fn().mockResolvedValue(undefined),
dispose: vi.fn(),
sessionManager: { getLeafId: vi.fn(), branchWithSummary: vi.fn() },
navigateTree: vi.fn(),
},
} as any;
});
const store = createMockStore();
store.getSettings.mockResolvedValue({ ...(await store.getSettings()), experimentalFeatures: { researchView: true } });
const executor = new TaskExecutor(store, "/tmp/test");
await executor.execute({
id: "FN-SYS-RESEARCH",
title: "Test",
description: "Test",
column: "in-progress",
dependencies: [],
steps: [],
currentStep: 0,
log: [],
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
});
expect(capturedSystemPrompt).toContain("fn_research_run");
});
it("EXECUTOR_SYSTEM_PROMPT contains code review enforcement language", async () => { it("EXECUTOR_SYSTEM_PROMPT contains code review enforcement language", async () => {
// Capture the system prompt passed to createFnAgent // Capture the system prompt passed to createFnAgent
let capturedSystemPrompt = ""; let capturedSystemPrompt = "";

View File

@@ -587,9 +587,9 @@ describe("buildSpecificationPrompt", () => {
}); });
describe("TRIAGE_SYSTEM_PROMPT", () => { describe("TRIAGE_SYSTEM_PROMPT", () => {
it("includes bounded research guidance", () => { it("does not include unconditional research guidance", () => {
expect(TRIAGE_SYSTEM_PROMPT).toContain("fn_research_run"); expect(TRIAGE_SYSTEM_PROMPT).not.toContain("fn_research_run");
expect(TRIAGE_SYSTEM_PROMPT).toContain("Keep research bounded"); expect(TRIAGE_SYSTEM_PROMPT).not.toContain("Keep research bounded");
}); });
it("requires specs to keep lint, tests, build, and typecheck green even outside initial file scope", () => { it("requires specs to keep lint, tests, build, and typecheck green even outside initial file scope", () => {
@@ -889,7 +889,7 @@ describe("fast-mode triage", () => {
} }
}); });
it("omits research tools when researchView experimental flag is disabled", async () => { it("omits research tools and prompt guidance when researchView experimental flag is disabled", async () => {
const task = createTriageTask({ id: "FN-FAST-005", executionMode: "fast" }); const task = createTriageTask({ id: "FN-FAST-005", executionMode: "fast" });
const store = createMockStore({ const store = createMockStore({
getSettings: vi.fn().mockResolvedValue({ getSettings: vi.fn().mockResolvedValue({
@@ -904,8 +904,10 @@ describe("fast-mode triage", () => {
}); });
let capturedTools: any[] = []; let capturedTools: any[] = [];
let capturedSystemPrompt = "";
mockCreateFnAgent.mockImplementationOnce(async (opts: any) => { mockCreateFnAgent.mockImplementationOnce(async (opts: any) => {
capturedTools = opts.customTools; capturedTools = opts.customTools;
capturedSystemPrompt = opts.systemPrompt;
return { return {
session: { session: {
state: {}, state: {},
@@ -924,6 +926,42 @@ describe("fast-mode triage", () => {
expect(capturedTools.some((tool: any) => tool.name === "fn_research_list")).toBe(false); expect(capturedTools.some((tool: any) => tool.name === "fn_research_list")).toBe(false);
expect(capturedTools.some((tool: any) => tool.name === "fn_research_get")).toBe(false); expect(capturedTools.some((tool: any) => tool.name === "fn_research_get")).toBe(false);
expect(capturedTools.some((tool: any) => tool.name === "fn_research_cancel")).toBe(false); expect(capturedTools.some((tool: any) => tool.name === "fn_research_cancel")).toBe(false);
expect(capturedSystemPrompt).not.toContain("fn_research_run");
});
it("includes research prompt guidance when researchView experimental flag is enabled", async () => {
const task = createTriageTask({ id: "FN-FAST-006", executionMode: "fast" });
const store = createMockStore({
getSettings: vi.fn().mockResolvedValue({
maxConcurrent: 2,
maxWorktrees: 4,
pollIntervalMs: 10000,
groupOverlappingFiles: false,
autoMerge: true,
experimentalFeatures: { researchView: true },
} as Settings),
getTask: vi.fn().mockResolvedValue({ ...mockTaskDetail, id: task.id, attachments: [], comments: [] }),
});
let capturedSystemPrompt = "";
mockCreateFnAgent.mockImplementationOnce(async (opts: any) => {
capturedSystemPrompt = opts.systemPrompt;
return {
session: {
state: {},
sessionManager: { getLeafId: vi.fn().mockReturnValue(null) },
prompt: vi.fn().mockResolvedValue(undefined),
dispose: vi.fn(),
navigateTree: vi.fn(),
},
};
});
const processor = new TriageProcessor(store, "/tmp/root");
await processor.specifyTask(task);
expect(capturedSystemPrompt).toContain("fn_research_run");
expect(capturedSystemPrompt).toContain("Keep research bounded");
}); });
}); });

View File

@@ -11,7 +11,6 @@ import {
buildExecutionMemoryInstructions, buildExecutionMemoryInstructions,
getTaskMergeBlocker, getTaskMergeBlocker,
isEphemeralAgent, isEphemeralAgent,
isResearchExperimentalEnabled,
resolveAgentPrompt, resolveAgentPrompt,
resolveEffectiveAgentPermissionPolicy, resolveEffectiveAgentPermissionPolicy,
resolveProjectDefaultModel, resolveProjectDefaultModel,
@@ -73,6 +72,11 @@ import {
createTaskLogTool as sharedCreateTaskLogTool, createTaskLogTool as sharedCreateTaskLogTool,
} from "./agent-tools.js"; } from "./agent-tools.js";
import { getTaskCompletionBlockerForStore } from "./task-completion.js"; import { getTaskCompletionBlockerForStore } from "./task-completion.js";
import {
getEnabledPluginTools,
getResearchGuidanceForSurface,
isResearchToolSurfaceEnabled,
} from "./tool-availability.js";
import { createFusionAuthStorage, getModelRegistryModelsPath } from "./auth-storage.js"; import { createFusionAuthStorage, getModelRegistryModelsPath } from "./auth-storage.js";
import { createRunVerificationTool } from "./run-verification-tool.js"; import { createRunVerificationTool } from "./run-verification-tool.js";
import { createFallbackModelObserver } from "./fallback-model-observer.js"; import { createFallbackModelObserver } from "./fallback-model-observer.js";
@@ -381,12 +385,6 @@ You can save and retrieve named documents for this task. Use these to store plan
Documents are versioned — each write creates a new revision. Use meaningful keys like "plan", "notes", "research", "architecture". Documents are versioned — each write creates a new revision. Use meaningful keys like "plan", "notes", "research", "architecture".
## Research tools
When implementation needs external context, you may use research tools (
\`fn_research_run\`, \`fn_research_list\`, \`fn_research_get\`, \`fn_research_cancel\`) to run bounded research.
Keep runs focused and short, and persist durable conclusions into task documents (for example key="research").
If research is disabled or providers are not configured, use the actionable tool response and continue with available local context.
**IMPORTANT — Save your deliverables as documents:** When your task produces written output (documentation, specifications, reports, API references, README updates, guides, or any other content), you MUST save that content as a task document using \`fn_task_document_write\`. Use a key that describes the deliverable (e.g., key="readme", key="api-docs", key="changelog"). Do this in addition to writing the file to disk — the document persists in the task for review even after the worktree is cleaned up. **IMPORTANT — Save your deliverables as documents:** When your task produces written output (documentation, specifications, reports, API references, README updates, guides, or any other content), you MUST save that content as a task document using \`fn_task_document_write\`. Use a key that describes the deliverable (e.g., key="readme", key="api-docs", key="changelog"). Do this in addition to writing the file to disk — the document persists in the task for review even after the worktree is cleaned up.
If the task's PROMPT.md includes a "Documentation Requirements" section listing files to update, save each updated file's final content as a task document with a matching key. If the task's PROMPT.md includes a "Documentation Requirements" section listing files to update, save each updated file's final content as a task document with a matching key.
@@ -503,7 +501,12 @@ The tool prevents your session from being killed by the inactivity watchdog duri
/** Resolve the executor system prompt from settings, falling back to the hardcoded constant. */ /** Resolve the executor system prompt from settings, falling back to the hardcoded constant. */
function getExecutorSystemPrompt(settings: Settings): string { function getExecutorSystemPrompt(settings: Settings): string {
const customPrompt = resolveAgentPrompt("executor", settings.agentPrompts); const customPrompt = resolveAgentPrompt("executor", settings.agentPrompts);
return customPrompt || EXECUTOR_SYSTEM_PROMPT; const basePrompt = customPrompt || EXECUTOR_SYSTEM_PROMPT;
const sections = [
basePrompt,
isResearchToolSurfaceEnabled(settings) ? getResearchGuidanceForSurface("executor") : "",
].filter((section) => section.trim());
return sections.join("\n\n");
} }
@@ -2880,7 +2883,7 @@ export class TaskExecutor {
this.createSpawnAgentTool(task.id, worktreePath, settings), this.createSpawnAgentTool(task.id, worktreePath, settings),
this.createTaskDocumentWriteTool(task.id), this.createTaskDocumentWriteTool(task.id),
this.createTaskDocumentReadTool(task.id), this.createTaskDocumentReadTool(task.id),
...(isResearchExperimentalEnabled(settings) ...(isResearchToolSurfaceEnabled(settings)
? createResearchTools({ ? createResearchTools({
store: this.store, store: this.store,
rootDir: this.rootDir, rootDir: this.rootDir,
@@ -2911,7 +2914,7 @@ export class TaskExecutor {
createReadMessagesTool(this.options.messageStore, assignedAgentId), createReadMessagesTool(this.options.messageStore, assignedAgentId),
] : []), ] : []),
// Add plugin tools from PluginRunner // Add plugin tools from PluginRunner
...(this.options.pluginRunner?.getPluginTools() ?? []), ...getEnabledPluginTools(this.options.pluginRunner),
]; ];
// Accumulates the full assistant text output for the most recent session. // Accumulates the full assistant text output for the most recent session.

View File

@@ -0,0 +1,38 @@
import type { Settings } from "@fusion/core";
import { isResearchExperimentalEnabled } from "@fusion/core";
import type { PluginRunner } from "./plugin-runner.js";
import type { ToolDefinition } from "@mariozechner/pi-coding-agent";
export function isResearchToolSurfaceEnabled(settings: Partial<Settings> | undefined): boolean {
return isResearchExperimentalEnabled(settings);
}
export function getResearchToolSurfaceStatus(settings: Partial<Settings> | undefined): {
enabled: boolean;
reason: "experimental-disabled" | "enabled";
} {
const enabled = isResearchToolSurfaceEnabled(settings);
return {
enabled,
reason: enabled ? "enabled" : "experimental-disabled",
};
}
const TRIAGE_RESEARCH_GUIDANCE = `## Research tools
When spec work needs missing domain context, you may use research tools (\`fn_research_run\`, \`fn_research_list\`, \`fn_research_get\`, \`fn_research_cancel\`). Keep research bounded to the task at hand, prefer concise queries, and write durable findings into task documents when useful.
If research is unavailable or unconfigured, continue planning with repository context and clearly note assumptions.`;
const EXECUTOR_RESEARCH_GUIDANCE = `## Research tools
When implementation needs external context, you may use research tools (
\`fn_research_run\`, \`fn_research_list\`, \`fn_research_get\`, \`fn_research_cancel\`) to run bounded research.
Keep runs focused and short, and persist durable conclusions into task documents (for example key="research").
If research is disabled or providers are not configured, use the actionable tool response and continue with available local context.`;
export function getResearchGuidanceForSurface(surface: "triage" | "executor"): string {
return surface === "triage" ? TRIAGE_RESEARCH_GUIDANCE : EXECUTOR_RESEARCH_GUIDANCE;
}
export function getEnabledPluginTools(pluginRunner: PluginRunner | undefined): ToolDefinition[] {
if (!pluginRunner) return [];
return pluginRunner.getPluginTools();
}

View File

@@ -8,7 +8,6 @@ import type {
} from "@fusion/core"; } from "@fusion/core";
import { import {
buildTriageMemoryInstructions, buildTriageMemoryInstructions,
isResearchExperimentalEnabled,
resolveAgentPrompt, resolveAgentPrompt,
sortTasksByPriorityThenAgeAndId, sortTasksByPriorityThenAgeAndId,
} from "@fusion/core"; } from "@fusion/core";
@@ -56,6 +55,10 @@ import {
createTaskDocumentReadTool, createTaskDocumentReadTool,
createTaskDocumentWriteTool, createTaskDocumentWriteTool,
} from "./agent-tools.js"; } from "./agent-tools.js";
import {
getResearchGuidanceForSurface,
isResearchToolSurfaceEnabled,
} from "./tool-availability.js";
export const TRIAGE_SYSTEM_PROMPT = `You are a task specification agent for "fn", an AI-orchestrated task board. export const TRIAGE_SYSTEM_PROMPT = `You are a task specification agent for "fn", an AI-orchestrated task board.
@@ -244,10 +247,6 @@ When the planning conversation produces a structured plan, save it as a document
- Testing & Verification must run before Documentation & Delivery - Testing & Verification must run before Documentation & Delivery
- Avoid giant catch-all steps; split outcomes so execution can be verified incrementally - Avoid giant catch-all steps; split outcomes so execution can be verified incrementally
## Research tools
When spec work needs missing domain context, you may use research tools (\`fn_research_run\`, \`fn_research_list\`, \`fn_research_get\`, \`fn_research_cancel\`). Keep research bounded to the task at hand, prefer concise queries, and write durable findings into task documents when useful.
If research is unavailable or unconfigured, continue planning with repository context and clearly note assumptions.
## Guidelines ## Guidelines
- Read the project structure and relevant source files to understand context BEFORE writing - Read the project structure and relevant source files to understand context BEFORE writing
- Check package.json/scripts and explicit project commands to align real lint/test/build/typecheck commands - Check package.json/scripts and explicit project commands to align real lint/test/build/typecheck commands
@@ -951,7 +950,7 @@ export class TriageProcessor {
}), }),
createTaskDocumentWriteTool(this.store, task.id), createTaskDocumentWriteTool(this.store, task.id),
createTaskDocumentReadTool(this.store, task.id), createTaskDocumentReadTool(this.store, task.id),
...(isResearchExperimentalEnabled(settings) ...(isResearchToolSurfaceEnabled(settings)
? createResearchTools({ ? createResearchTools({
store: this.store, store: this.store,
rootDir: this.rootDir, rootDir: this.rootDir,
@@ -1016,7 +1015,13 @@ export class TriageProcessor {
const triageSystemPrompt = buildSystemPromptWithInstructions( const triageSystemPrompt = buildSystemPromptWithInstructions(
resolveAgentPrompt("triage", settings.agentPrompts) resolveAgentPrompt("triage", settings.agentPrompts)
|| (isFast ? FAST_TRIAGE_SYSTEM_PROMPT : TRIAGE_SYSTEM_PROMPT), || (isFast ? FAST_TRIAGE_SYSTEM_PROMPT : TRIAGE_SYSTEM_PROMPT),
[triageIdentitySection, triageInstructions].filter((section) => section.trim()).join("\n\n"), [
triageIdentitySection,
triageInstructions,
isResearchToolSurfaceEnabled(settings)
? getResearchGuidanceForSurface("triage")
: "",
].filter((section) => section.trim()).join("\n\n"),
); );
const triageContributions = this.options.pluginRunner const triageContributions = this.options.pluginRunner
?.getPromptContributionsForSurface("triage") ?.getPromptContributionsForSurface("triage")