root/fusion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

FN-8331: require planning interview before summaries

Browse Source 
Ensure Planning Mode collects mandatory user context before it can generate a plan.

- Reject and re-prompt first-turn completion responses until a clarifying question is available.
- Preserve the interview invariant for streaming sessions and legacy session rehydration.
- Update Planning Mode defaults, documentation, regression coverage, and release metadata.

Files changed:
 .changeset/fn-8331-planning-interview.md           |   7 +
 docs/dashboard-guide.md                            |   2 +-
 docs/settings-reference.md                         |   2 +-
 .../dashboard/app/components/PlanningModeModal.tsx |  10 +-
 .../PlanningModeModal.planning-flow.test.tsx       |   4 +-
 .../src/__tests__/routes-planning.test.ts          | 120 ++++++++++++++++-
 packages/dashboard/src/planning.ts                 | 145 ++++++++++++++-------
 7 files changed, 233 insertions(+), 57 deletions(-)

Fusion-Task-Id: FN-8331

Fusion-Task-Lineage: 6fc98f2e-f263-4830-b88c-e52d1ba6551f

Co-authored-by: Fusion (runfusion.ai) <noreply@runfusion.ai>
This commit is contained in:
gsxdsm
2026-07-18 13:01:35 -07:00
parent 500275ac0c
commit 551074537f
7 changed files with 233 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.changeset/fn-8331-planning-interview.md Normal file
View File

@@ -0,0 +1,7 @@
---
"@runfusion/fusion": patch
---
summary: Planning Mode now always asks clarifying questions before producing a plan.
category: fix
dev: createSession/processAgentTurn reject a first-turn completion and no longer suppress the first clarifying question when clarification is disabled.

2
docs/dashboard-guide.md
View File

@@ -2075,7 +2075,7 @@ If the endpoint is unavailable on the running dashboard build, the response will
### Planner clarification notifications
In **Settings → Notifications**, enable **Agent clarification** to let Planning Mode pause when the planner needs an answer. The Planning Mode advanced settings include a per-session override, initialized from that global preference. With clarification disabled, proactive questions are redirected to a final plan summary instead of holding the session; the final summary deepening checkpoint is unchanged.
Planning Mode always asks and waits for at least one clarifying question before producing a plan. The advanced **follow-up clarification questions** setting is a per-session override initialized from the global preference: when disabled, Planning Mode still asks one mandatory question, then requests a final summary after the answer; when enabled, it may ask further proactive questions. The final summary deepening checkpoint is unchanged.
When enabled, a proactive question holds the planner at `awaiting_input`, sends the configured `planning-awaiting-input` ntfy event, and delivers a dashboard mailbox message that links the operator back to planner chat. Mailbox delivery does not depend on ntfy configuration and is deduplicated by session/question across restarts.

2
docs/settings-reference.md
View File

@@ -71,7 +71,7 @@ Fallback thinking-level values are applied at runtime when Fusion swaps from the
| `defaultThinkingLevel` | `"off" \| "minimal" \| "low" \| "medium" \| "high" \| "xhigh"` | `undefined` | Default reasoning effort for AI sessions. `xhigh` requests maximum reasoning effort; Claude CLI adapters map it to `high` for non-Opus models and `max` for Opus models. If a provider/runtime rejects simultaneous `thinking` and `reasoning_effort` parameters, Fusion retries without the explicit thinking override instead of failing the run. |
| `ntfyEnabled` | `boolean` | `false` | Enable ntfy push notifications. |
| `agentClarificationEnabled` | `boolean` | `false` | Allow Planning Mode to pause for proactive AI clarification questions. When disabled, the planner requests a final summary instead; enabled proactive questions notify configured ntfy recipients and the dashboard mailbox. |
| `agentClarificationEnabled` | `boolean` | `false` | Allow follow-up Planning Mode clarification questions. Planning Mode always asks one mandatory question first; when disabled it requests a final summary after that answer, while enabled follow-ups notify configured ntfy recipients and the dashboard mailbox. |
| `failureNotificationMode` | `"sticky-only" \| "terminal-only" \| "all"` | `"sticky-only"` | Failure notification behavior. `sticky-only` defers failed-task notifications by `failureNotificationDelayMs` and suppresses transient self-recoveries. `terminal-only` suppresses while auto-retry is still active and only dispatches when `paused === true` or `column === "in-review"` with `status === "failed"`. `all` restores legacy immediate failure notifications. |
| `failureNotificationDelayMs` | `number` | `30000` | Delay window (ms) before evaluating/sending a `failed` notification in `sticky-only` and `terminal-only` modes. Set `0` for immediate dispatch in legacy `all` mode. |
| `ntfyTopic` | `string` | `undefined` | ntfy topic name. |

10
packages/dashboard/app/components/PlanningModeModal.tsx
View File

@@ -438,7 +438,7 @@ export function PlanningModeModal({ isOpen, onClose, onTaskCreated, onTasksCreat
const [planningThinkingLevel, setPlanningThinkingLevel] = useState<ThinkingLevel | "">("");
const [planningDepth, setPlanningDepth] = useState<"small" | "medium" | "large">("medium");
const [customQuestionCount, setCustomQuestionCount] = useState("");
const [clarificationEnabled, setClarificationEnabled] = useState(false);
const [clarificationEnabled, setClarificationEnabled] = useState(true);
const [clarificationSettingsLoading, setClarificationSettingsLoading] = useState(true);
const [loadedModels, setLoadedModels] = useState<ModelInfo[]>([]);
const [modelsLoading, setModelsLoading] = useState(false);
@@ -1171,9 +1171,9 @@ export function PlanningModeModal({ isOpen, onClose, onTaskCreated, onTasksCreat
let active = true;
setClarificationSettingsLoading(true);
void fetchGlobalSettings()
.then((settings) => { if (active) setClarificationEnabled(settings.agentClarificationEnabled === true); })
// Safe fallback keeps automatic/manual starts from unexpectedly pausing.
.catch(() => { if (active) setClarificationEnabled(false); })
.then((settings) => { if (active) setClarificationEnabled(settings.agentClarificationEnabled !== false); })
// A missing settings response keeps the default full interview; disabled only limits follow-ups.
.catch(() => { if (active) setClarificationEnabled(true); })
.finally(() => { if (active) setClarificationSettingsLoading(false); });
return () => { active = false; };
}, [isOpen]);
@@ -2400,7 +2400,7 @@ export function PlanningModeModal({ isOpen, onClose, onTaskCreated, onTasksCreat
<div className="planning-advanced-section planning-depth-selector">
<label className="checkbox-label" htmlFor="planning-clarification-enabled">
<input id="planning-clarification-enabled" type="checkbox" checked={clarificationEnabled} disabled={clarificationSettingsLoading} onChange={(event) => setClarificationEnabled(event.target.checked)} />
{t("planning.agentClarification", " Allow agent clarification questions")}
{t("planning.agentClarification", " Allow follow-up clarification questions")}
</label>
<p className="planning-advanced-blurb">
{t("planning.depthBlurb", "Plan size sets default interview depth. Questions lets you override with an exact count.")}

4
packages/dashboard/app/components/__tests__/PlanningModeModal.planning-flow.test.tsx
View File

@@ -478,7 +478,7 @@ describe("PlanningModeModal", () => {
expect(mockStartPlanningStreaming).toHaveBeenCalledWith("Build auth system", undefined, undefined, {
planningDepth: "medium",
customQuestionCount: undefined,
clarificationEnabled: false,
clarificationEnabled: true,
}, undefined);
});
@@ -2253,7 +2253,7 @@ describe("PlanningModeModal", () => {
"Plan that needs a specific model",
undefined,
{ planningModelProvider: "anthropic", planningModelId: "claude-sonnet-4-5", thinkingLevel: undefined },
{ planningDepth: "medium", customQuestionCount: undefined, clarificationEnabled: false },
{ planningDepth: "medium", customQuestionCount: undefined, clarificationEnabled: true },
"session-draft-with-model",
);
});

120
packages/dashboard/src/__tests__/routes-planning.test.ts
View File

@@ -563,6 +563,69 @@ describe("Planning Mode Routes", () => {
expect(res.body.firstQuestion.type).toBe("single_select");
});
it("rejects a first-turn completion until the agent asks a clarifying question", async () => {
const messages: Array<{ role: string; content: string }> = [];
const responses = [
JSON.stringify({ type: "complete", data: { title: "Too early", description: "A plan", keyDeliverables: [] } }),
JSON.stringify({ type: "question", data: { id: "q-required", type: "text", question: "Which constraint matters most?" } }),
];
let responseIndex = 0;
__setCreateFnAgent(async () => ({
session: {
state: { messages },
prompt: vi.fn(async (message: string) => {
messages.push({ role: "user", content: message });
messages.push({ role: "assistant", content: responses[responseIndex++]! });
}),
dispose: vi.fn(),
},
}));
const res = await REQUEST(
buildApp(),
"POST",
"/api/planning/start",
JSON.stringify({ initialPlan: "Build a detailed account-management experience" }),
{ "Content-Type": "application/json" },
);
expect(res.status).toBe(201);
expect(res.body.firstQuestion).toMatchObject({ id: "q-required", question: "Which constraint matters most?" });
expect(res.body.firstQuestion.id).not.toBe(PLANNING_DEEPEN_CHECKPOINT_ID);
expect(messages).toHaveLength(4);
expect(messages[2]?.content).toContain("Before producing a plan");
});
it("shows the mandatory first question when clarification is disabled", async () => {
const messages: Array<{ role: string; content: string }> = [];
__setCreateFnAgent(async () => ({
session: {
state: { messages },
prompt: vi.fn(async (message: string) => {
messages.push({ role: "user", content: message });
messages.push({ role: "assistant", content: JSON.stringify({
type: "question",
data: { id: "q-required-disabled", type: "text", question: "Who is the primary user?" },
}) });
}),
dispose: vi.fn(),
},
}));
const res = await REQUEST(
buildApp(),
"POST",
"/api/planning/start",
JSON.stringify({ initialPlan: "Build a feature", clarificationEnabled: false }),
{ "Content-Type": "application/json" },
);
expect(res.status).toBe(201);
expect(res.body.firstQuestion).toMatchObject({ id: "q-required-disabled" });
expect(res.body.firstQuestion.id).not.toBe(PLANNING_DEEPEN_CHECKPOINT_ID);
expect(messages).toHaveLength(2);
});
it("requires initialPlan in body", async () => {
const res = await REQUEST(
buildApp(),
@@ -660,6 +723,44 @@ describe("Planning Mode Routes", () => {
});
describe("POST /planning/start-streaming", () => {
it("broadcasts a mandatory question instead of accepting a first-turn completion", async () => {
const messages: Array<{ role: string; content: string }> = [];
const responses = [
JSON.stringify({ type: "complete", data: { title: "Too early", description: "A plan", keyDeliverables: [] } }),
JSON.stringify({ type: "question", data: { id: "q-stream-required", type: "text", question: "What risk should the plan address?" } }),
];
let responseIndex = 0;
__setCreateFnAgent(async () => ({
session: {
state: { messages },
prompt: vi.fn(async (message: string) => {
messages.push({ role: "user", content: message });
messages.push({ role: "assistant", content: responses[responseIndex++]! });
}),
dispose: vi.fn(),
},
}));
const res = await REQUEST(
buildApp(),
"POST",
"/api/planning/start-streaming",
JSON.stringify({ initialPlan: "Build a detailed reporting workflow", clarificationEnabled: false, planningDepth: "small", customQuestionCount: 1 }),
{ "Content-Type": "application/json" },
);
expect(res.status).toBe(201);
planningStreamManager.consumeInitialTurn(res.body.sessionId)!();
await vi.waitFor(() => {
const questions = planningStreamManager.getBufferedEvents(res.body.sessionId, 0)
.filter((event) => event.event === "question");
expect(questions).toHaveLength(1);
expect(JSON.parse(questions[0]!.data)).toMatchObject({ id: "q-stream-required" });
});
expect(messages).toHaveLength(4);
expect(messages[2]?.content).toContain("Before producing a plan");
});
it("rejects invalid planning depth", async () => {
const res = await REQUEST(
buildApp(),
@@ -1637,6 +1738,10 @@ describe("Planning Mode Routes", () => {
it("prefers AI-authored deepeningThemes over generic themes on both completion paths", async () => {
const responses = [
JSON.stringify({
type: "question",
data: { id: "q-offline-context", type: "text", question: "Which offline scenarios matter most?" },
}),
JSON.stringify({
type: "complete",
data: {
@@ -1673,9 +1778,18 @@ describe("Planning Mode Routes", () => {
);
const sessionId = startRes.body.sessionId;
expect(startRes.body.firstQuestion.question).toBe(PLANNING_DEEPEN_CHECKPOINT_QUESTION);
expect(startRes.body.firstQuestion.options?.[0]?.id).toBe(PLANNING_DEEPEN_PROCEED_OPTION_ID);
expect(startRes.body.firstQuestion.options?.map((option: { label: string }) => option.label)).toEqual([
expect(startRes.body.firstQuestion.question).toBe("Which offline scenarios matter most?");
const interviewRes = await REQUEST(
buildApp(),
"POST",
"/api/planning/respond",
JSON.stringify({ sessionId, responses: { "q-offline-context": "Conflicts and recovery" } }),
{ "Content-Type": "application/json" },
);
expect(interviewRes.body.data.question).toBe(PLANNING_DEEPEN_CHECKPOINT_QUESTION);
expect(interviewRes.body.data.options?.[0]?.id).toBe(PLANNING_DEEPEN_PROCEED_OPTION_ID);
expect(interviewRes.body.data.options?.map((option: { label: string }) => option.label)).toEqual([
"Proceed to final plan",
"Conflict resolution strategy",
]);

145
packages/dashboard/src/planning.ts
View File

@@ -903,12 +903,32 @@ function buildSessionFromRow(row: AiSessionRow): Session {
pre-fix builds still carry the already-answered question; restoring it would let the SSE
catch-up path re-emit it and re-trigger the answered-question retry loop after a restart.
*/
const currentQuestion = row.status === "awaiting_input" && row.currentQuestion
? (safeParseJson<PlanningQuestion | null>(row.currentQuestion, null, {
throwOnError: true,
fieldName: "currentQuestion",
}) ?? undefined)
const history = safeParseJson<PlanningHistoryEntry[]>(
row.conversationHistory,
[],
{ throwOnError: true, fieldName: "conversationHistory" },
);
const persistedSummary = row.result
? normalizePlanningSummaryPayload(
safeParseJson<unknown | null>(row.result, null, {
throwOnError: true,
fieldName: "result",
}),
{ title: row.title, description: row.title },
)
: undefined;
const persistedPendingSummary = payload.pendingSummary
? normalizePlanningSummaryPayload(payload.pendingSummary, { title: row.title, description: row.title })
: undefined;
const skippedMandatoryInterview = history.length === 0 && Boolean(persistedSummary || persistedPendingSummary);
const currentQuestion = skippedMandatoryInterview
? buildMandatoryFirstPlanningQuestion()
: row.status === "awaiting_input" && row.currentQuestion
? (safeParseJson<PlanningQuestion | null>(row.currentQuestion, null, {
throwOnError: true,
fieldName: "currentQuestion",
}) ?? undefined)
: undefined;
return {
id: row.id,
@@ -926,25 +946,11 @@ function buildSessionFromRow(row: AiSessionRow): Session {
lastMailboxNotifiedQuestionKey: typeof payload.lastMailboxNotifiedQuestionKey === "string"
? payload.lastMailboxNotifiedQuestionKey
: undefined,
history: safeParseJson<PlanningHistoryEntry[]>(
row.conversationHistory,
[],
{ throwOnError: true, fieldName: "conversationHistory" },
),
history,
currentQuestion,
lastNotifiedQuestionKey: currentQuestion ? `${row.id}:${currentQuestion.id}` : undefined,
summary: row.result
? normalizePlanningSummaryPayload(
safeParseJson<unknown | null>(row.result, null, {
throwOnError: true,
fieldName: "result",
}),
{ title: row.title, description: row.title },
)
: undefined,
pendingSummary: payload.pendingSummary
? normalizePlanningSummaryPayload(payload.pendingSummary, { title: row.title, description: row.title })
: undefined,
summary: skippedMandatoryInterview ? undefined : persistedSummary,
pendingSummary: skippedMandatoryInterview ? undefined : persistedPendingSummary,
thinkingOutput: row.thinkingOutput,
lastGeneratedThinking: row.thinkingOutput || "",
error: row.error ?? undefined,
@@ -969,6 +975,10 @@ export async function rehydrateFromStore(store: AiSessionStore): Promise<number>
try {
const session = buildSessionFromRow(row);
sessions.set(session.id, session);
if (session.currentQuestion && session.history.length === 0 && (row.result || safeParseJson<DraftInputPayload>(row.inputPayload, {}).pendingSummary)) {
/* FNXC:PlanningMode 2026-07-18-11:36: Rehydration repairs legacy no-history summaries into the mandatory interview question so reconnects cannot revive a skipped interview. */
persistSession(session, "awaiting_input");
}
rehydrated += 1;
} catch (error) {
diagnostics.errorFromException("Failed to rehydrate session", error, { sessionId: row.id, operation: "rehydrate" });
@@ -1315,25 +1325,6 @@ export async function createSession(
// Send initial plan to get first question from AI
const firstResponse = await getFirstQuestionFromAgent(session, initialPlan);
if (firstResponse.type === "complete") {
const firstQuestion = setPendingSummaryCheckpoint(session, normalizePlanningSummaryPayload(firstResponse.data, {
title: session.title || session.initialPlan,
description: session.initialPlan,
}));
return { sessionId, firstQuestion };
}
if (!session.clarificationEnabled) {
try {
const firstQuestion = await continueToSummaryAfterSuppressedQuestion(session);
return { sessionId, firstQuestion };
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
setSessionError(session, message);
throw error;
}
}
const firstQuestion = firstResponse.data;
session.currentQuestion = firstQuestion;
session.updatedAt = new Date();
@@ -1350,8 +1341,8 @@ export async function createSession(
*/
async function getFirstQuestionFromAgent(
session: Session,
message: string
): Promise<PlanningResponse> {
message: string,
): Promise<{ type: "question"; data: PlanningQuestion }> {
if (!session.agent) {
throw new InvalidSessionStateError("AI agent not initialized");
}
@@ -1474,7 +1465,58 @@ async function getFirstQuestionFromAgent(
throw new Error(`Failed to get first question from AI: ${errorMessage}`);
}
return parsed;
if (parsed.type === "question") {
return parsed;
}
/*
FNXC:PlanningMode 2026-07-18-11:36:
FN-8331 makes the first planning turn an interview invariant: a completion cannot become a
deepening checkpoint until the user has answered a real clarifying question. Re-prompt once
for the required protocol shape, then use a safe local question if the model still refuses.
*/
return requestMandatoryFirstPlanningQuestion(session);
}
function buildMandatoryFirstPlanningQuestion(): PlanningQuestion {
return {
id: "mandatory-planning-clarification",
type: "text",
question: "What outcome or constraint is most important for this plan?",
description: "Answer this first question so the plan can reflect your priorities.",
};
}
async function requestMandatoryFirstPlanningQuestion(
session: Session,
abortSignal?: AbortSignal,
): Promise<{ type: "question"; data: PlanningQuestion }> {
try {
await (session.agent!.session.prompt as (input: string, options?: { signal?: AbortSignal }) => Promise<void>)(
'Before producing a plan, ask one clarifying question. Return ONLY valid JSON: {"type":"question","data":{...}}.',
{ signal: abortSignal },
);
const retryMessage = (session.agent!.session.state.messages as AgentMessage[])
.filter((m) => m.role === "assistant")
.pop();
const retryText = typeof retryMessage?.content === "string"
? retryMessage.content
: Array.isArray(retryMessage?.content)
? retryMessage.content
.filter((block): block is { type: "text"; text: string } => block.type === "text" && typeof block.text === "string")
.map((block) => block.text)
.join("")
: "";
const retryResponse = parseAgentResponse(retryText);
if (retryResponse.type === "question") return retryResponse;
} catch (error) {
diagnostics.warn("Agent did not supply the mandatory first planning question", {
sessionId: session.id,
message: error instanceof Error ? error.message : String(error),
operation: "mandatory-first-question",
});
}
return { type: "question", data: buildMandatoryFirstPlanningQuestion() };
}
/**
@@ -2507,7 +2549,7 @@ async function continueAgentConversation(session: Session, message: string): Pro
session.error = undefined;
session.lastGeneratedThinking = session.thinkingOutput;
session.updatedAt = new Date();
if (!session.clarificationEnabled) {
if (!session.clarificationEnabled && session.history.length > 0) {
try {
await continueToSummaryAfterSuppressedQuestion(session, abortSignal);
} catch (error) {
@@ -2519,6 +2561,19 @@ async function continueAgentConversation(session: Session, message: string): Pro
void maybeNotifyPlanningAwaitingInput(session, parsed.data, true);
planningStreamManager.broadcast(session.id, { type: "question", data: parsed.data });
} else if (parsed.type === "complete") {
if (session.history.length === 0) {
const mandatoryQuestion = await requestMandatoryFirstPlanningQuestion(session, abortSignal);
session.currentQuestion = mandatoryQuestion.data;
session.summary = undefined;
session.pendingSummary = undefined;
session.error = undefined;
session.lastGeneratedThinking = session.thinkingOutput;
session.updatedAt = new Date();
persistSession(session, "awaiting_input");
void maybeNotifyPlanningAwaitingInput(session, mandatoryQuestion.data, true);
planningStreamManager.broadcast(session.id, { type: "question", data: mandatoryQuestion.data });
return;
}
const summary = normalizePlanningSummaryPayload(parsed.data, {
title: session.title || session.initialPlan,
description: session.initialPlan,