feat(FN-871): implement deterministic fuzzy matching for model filter

- Rewrite filterModels utility with deterministic fuzzy matching (no probabilistic ranking)
- Extract model filtering logic from TaskDetailModal into dedicated modelFilter.ts utility
- Add comprehensive regression test suite for fuzzy matching edge cases
- Remove inline model filter tests from TaskDetailModal test file
- Clean up unused CSS styles related to old model filter UI
- Document fuzzy model search behavior in dashboard README
This commit is contained in:
gsxdsm
2026-04-04 07:36:07 -07:00
parent 8a2ee4a23c
commit 158002c6d6
3 changed files with 338 additions and 4 deletions

View File

@@ -49,6 +49,7 @@ AI-guided interactive planning for creating well-specified tasks from high-level
- **Model Selection at Creation**: Choose executor and validator AI models while creating tasks from the board or list view, or leave them unset to use the global defaults. Quick-add model dropdowns in both the board triage column and the list view honor saved favorite providers and pinned models, matching the rest of the dashboard model UI. Saved model presets are available in every new-task model-selection surface — the full New Task form (`TaskForm`/`NewTaskModal`), the board inline create card (`InlineCreateCard`), and the list-view quick entry box (`QuickEntryBox`). Users can choose between default behavior, a saved preset (which applies its executor/validator values in one click), or custom per-model overrides; returning to default clears all overrides, and manual model selection exits preset mode cleanly.
- **AI-Assisted Creation Controls**: Plan, Subtask, and Refine buttons appear directly below the description textarea in all task creation surfaces (quick entry box, inline create card, and task form modal). These description-adjacent controls make AI-assisted creation and refinement feel directly associated with the text being edited. Deps, Models, and Save actions remain in the expanded controls footer.
- **Layered Model Dropdowns**: Shared model combobox menus render in a top-level portal attached to `document.body`, so they stay above board columns and scrollable modal content instead of being clipped behind surrounding dashboard surfaces.
- **Fuzzy Model Search**: Model dropdown search (`filterModels`) supports fuzzy matching so users can find models despite minor typing imperfections. Three matching strategies are applied in order (first match wins): (1) **separator-insensitive substring** — hyphens, underscores, dots, and slashes are stripped before comparison, so `gpt4o` finds `gpt-4o`; (2) **subsequence matching** (≥ 3 chars) — characters must appear in order within a single token but need not be contiguous, so `cld` finds `claude`; (3) **typo tolerance** (≥ 4 chars) — Damerau-Levenshtein edit distance ≤ 1 supports single-character insertion, deletion, substitution, and adjacent transposition, so `sonet` finds `sonnet`. Multi-term space-separated queries use AND logic. Result ordering is stable (input-array order, no score re-sorting). Exact and substring matches from the original implementation continue to work unchanged.
- **Bulk Model Editing**: Update AI model configuration for multiple tasks at once in the list view. Select tasks via checkboxes (archived tasks excluded), then use the "Bulk Edit Models" toolbar to apply executor and/or validator model changes to all selected tasks. Selection persists in localStorage across page reloads.
- **Task Details**: View full task specifications, agent logs, and attachments. The Agent Log tab expands to fill the full modal body height above the action bar, providing maximum vertical space for watching live agent output. The tab header shows the effective executor and validator model names resolved from task-level overrides or project/global settings fallbacks, matching the same resolution order the engine uses at runtime. A **Markdown/Plain toggle** in the header bar switches between formatted markdown rendering (default) and literal plain-text display — useful for debugging raw agent output, checking escaped markdown syntax, or inspecting exactly what the agent emitted without formatting. The toggle applies to `text` and `thinking` entries only; tool entries always render as plain text. React-markdown handles sanitization in markdown mode (no raw HTML is executed); plain-text mode uses React's built-in text escaping for safe literal output. The refinement modal positions the "Create Refinement Task" button adjacent to the feedback textarea alongside the character count, creating a tight input group that connects the submit action directly to the text being edited. The **Changes** tab for done tasks loads the diff from the recorded merge commit (`mergeDetails.commitSha`) via `fetchCommitDiff` rather than requiring a live worktree — changes remain visible even after the worktree is cleaned up. The tab shows commit metadata (short SHA, merge commit message, merged timestamp) alongside the file-level diff with addition/deletion totals. In-progress and in-review tasks continue to use the worktree-based diff path.
- **Changed Files Viewer**: Click a task card's "files changed" button to open a dedicated diff viewer showing only files changed in that task worktree, with per-file statuses and sidebar navigation. On mobile (≤768px), the viewer switches to a single-pane flow: the file list and diff are shown one at a time with a back button for navigation between them. The viewer always opens to the file list on mobile, and only switches to the diff view when the user taps a specific file. Pressing Escape on the diff view returns to the file list first; pressing Escape again closes the modal. Loading, error, and empty states use theme-aware styling (including light mode). Diff syntax highlighting (additions, deletions, hunks) adapts to the active theme for correct contrast. The board card file count and the changed-files viewer always agree — both use a shared diff-base resolution strategy. When the task has a `baseCommitSha` (captured at worktree creation time) that is still a valid ancestor of the current HEAD, the diff is scoped to only files introduced by that specific task. If `baseCommitSha` is stale or unavailable, the system falls back to a branch merge-base, then to `HEAD~1`. This ensures accurate file counts in shared or recycled worktree scenarios where a broader merge-base would include files from previous tasks.

View File

@@ -163,4 +163,202 @@ describe("filterModels", () => {
const resultLl = filterModels(models, "ll");
expect(resultLl.map((m) => m.id)).toContain("llama3.1");
});
// --- Fuzzy matching: separator-insensitive ---
describe("separator-insensitive matching", () => {
it("matches when search omits hyphens from model ID", () => {
// "gpt4o" should match "gpt-4o" (hyphen omitted)
const result = filterModels(models, "gpt4o");
expect(result).toHaveLength(2);
expect(result.map((m) => m.id)).toContain("gpt-4o");
expect(result.map((m) => m.id)).toContain("gpt-4o-mini");
});
it("matches when search omits dots from model ID", () => {
const modelsWithDots = [
createModel("ollama", "llama3.1", "Llama 3.1"),
];
// "llama31" should match "llama3.1" (dot omitted)
expect(filterModels(modelsWithDots, "llama31")).toHaveLength(1);
});
it("matches when search omits underscores", () => {
const modelsWithUnderscores = [
createModel("test", "my_model_v2", "My Model V2"),
];
expect(filterModels(modelsWithUnderscores, "mymodelv2")).toHaveLength(1);
});
it("matches when search uses different separators than the model ID", () => {
// Searching with hyphen where the ID uses dot should still match
const result = filterModels(models, "gpt-4o");
expect(result).toHaveLength(2);
});
});
// --- Fuzzy matching: typo tolerance ---
describe("typo-tolerant matching", () => {
it("matches with single character deletion (sonet → sonnet)", () => {
const result = filterModels(models, "sonet");
expect(result).toHaveLength(1);
expect(result[0].id).toBe("claude-sonnet-4-5");
});
it("matches with single character insertion", () => {
// "sonnnet" (extra n) should still match "sonnet"
const result = filterModels(models, "sonnnet");
expect(result).toHaveLength(1);
expect(result[0].id).toBe("claude-sonnet-4-5");
});
it("matches with single character substitution", () => {
// "gemeno" → one substitution from "gemini" is too far, but "gemini" is close
// "gemini" with 'n' instead of 'i' at end → "geminj" should match
// Actually let's use a clear case: "gemino" (o instead of i) matches "gemini"
const result = filterModels(models, "gemino");
expect(result).toHaveLength(1);
expect(result[0].id).toBe("gemini-pro");
});
it("matches with adjacent transposition", () => {
// "opneai" (transposed n and e) should match "openai"
const result = filterModels(models, "opneai");
expect(result).toHaveLength(2); // Both openai models
});
it("does not apply typo tolerance to very short terms (≤ 3 chars)", () => {
// "xai" should NOT match "openai" via typo tolerance (edit distance 1)
// because the term is only 3 chars — fuzzy matching requires ≥ 4 chars
const result = filterModels(models, "xai");
// "xai" is not a substring, not a subsequence of any single token
expect(result).toEqual([]);
});
it("preserves multi-term AND logic with typo-tolerant terms", () => {
// "anthropic sonet" → "anthropic" matches exactly, "sonet" fuzzy-matches "sonnet"
const result = filterModels(models, "anthropic sonet");
expect(result).toHaveLength(1);
expect(result[0].id).toBe("claude-sonnet-4-5");
});
it("does not match when both terms are required but only one fuzzy-matches", () => {
// "google sonet" → "google" matches, "sonet" doesn't match any google model
const result = filterModels(models, "google sonet");
expect(result).toEqual([]);
});
});
// --- Fuzzy matching: subsequence (non-contiguous) ---
describe("subsequence matching", () => {
it("matches non-contiguous characters (cld → claude)", () => {
const result = filterModels(models, "cld");
// "cld" is a subsequence of "claude" (token), should match all claude models
expect(result).toHaveLength(2);
expect(result.map((m) => m.id)).toContain("claude-sonnet-4-5");
expect(result.map((m) => m.id)).toContain("claude-opus-4");
});
it("matches non-contiguous characters in model name", () => {
// "gmi" is a subsequence of "gemini" (g-e-m-i-n-i → g(0), m(2), i(3))
// It's also a subsequence of "gpt4omini" (g(0), m(5), i(6))
const result = filterModels(models, "gmi");
expect(result).toHaveLength(2);
expect(result.map((m) => m.id)).toContain("gemini-pro");
expect(result.map((m) => m.id)).toContain("gpt-4o-mini");
});
it("does not apply subsequence matching for very short terms (< 3 chars)", () => {
// "op" is 2 chars, so subsequence matching does NOT apply (min 3).
// However, "op" IS a substring: it appears in "anthropic" ("anthr**op**ic")
// and in "openai" ("**op**enai"), so it matches all 4 models from those providers.
const result = filterModels(models, "op");
expect(result).toHaveLength(4);
expect(result.map((m) => m.id)).toContain("claude-sonnet-4-5");
expect(result.map((m) => m.id)).toContain("claude-opus-4");
expect(result.map((m) => m.id)).toContain("gpt-4o");
expect(result.map((m) => m.id)).toContain("gpt-4o-mini");
});
it("requires all characters in order for subsequence", () => {
// "dcl" is NOT a subsequence of "claude" (d before c, but "dcl" reversed)
const result = filterModels(models, "dcl");
expect(result).toEqual([]);
});
it("subsequence only matches within individual tokens, not across fields", () => {
// "ops" should NOT match by picking 'o' from one field and 'ps' from another
// It should only match if it's a subsequence of a single token
// "ops" as subsequence of "claudeopus4" → o at index 6, p at index 7, s at index 9 → TRUE
// So it DOES match the opus model because it's a subsequence of the token "claudeopus4"
const result = filterModels(models, "ops");
expect(result).toHaveLength(1);
expect(result[0].id).toBe("claude-opus-4");
});
it("subsequence does not match across space-separated tokens", () => {
// "cpo" picking c from "claude", p from provider "anthropic", o from "4"
// should NOT match because subsequence is checked per-token
// "cpo" is NOT a subsequence of any single token
const result = filterModels(models, "cpo");
expect(result).toEqual([]);
});
});
// --- Fuzzy matching: negative tests (no over-matching) ---
describe("negative fuzzy matching (no over-matching)", () => {
it("returns empty array for clearly irrelevant input", () => {
expect(filterModels(models, "xyz")).toEqual([]);
expect(filterModels(models, "banana")).toEqual([]);
expect(filterModels(models, "zzzzz")).toEqual([]);
});
it("does not fuzzy-match unrelated providers", () => {
// "googel" is close to "google" (edit distance 1) but NOT to "openai" or "anthropic"
const result = filterModels(models, "googel");
expect(result).toHaveLength(1);
expect(result[0].provider).toBe("google");
});
it("does not fuzzy-match when edit distance exceeds tolerance", () => {
// "gpt5o" has edit distance 2 from "gpt4o" (4→5 substitution + different letter)
// Actually edit distance is 1 (just 4→5). Let's use a clear 2-distance case.
// "gpt99" has edit distance ≥ 2 from "gpt4o" (two substitutions: 4→9, o→9)
expect(filterModels(models, "gpt99")).toEqual([]);
});
it("does not fuzzy-match very different words", () => {
// "elephant" should not match anything despite fuzzy matching
expect(filterModels(models, "elephant")).toEqual([]);
});
it("multi-term AND with one non-matching term returns empty", () => {
// Even if "sonet" fuzzy-matches, adding "elephant" should return empty
expect(filterModels(models, "sonet elephant")).toEqual([]);
});
});
// --- Fuzzy matching: result ordering stability ---
describe("result ordering", () => {
it("preserves input-array order (no fuzzy-score re-sorting)", () => {
// All claude models should appear in their original array order
const result = filterModels(models, "claude");
expect(result.map((m) => m.id)).toEqual([
"claude-sonnet-4-5",
"claude-opus-4",
]);
});
it("preserves input-array order with fuzzy matches", () => {
const result = filterModels(models, "gpt4o");
expect(result.map((m) => m.id)).toEqual([
"gpt-4o",
"gpt-4o-mini",
]);
});
});
});

View File

@@ -1,15 +1,150 @@
import type { ModelInfo } from "../api";
/**
* Normalize a string for fuzzy matching:
* - Lowercase
* - Remove separator characters (hyphen, underscore, dot, slash)
*
* Preserves spaces (which serve as field/word boundaries) and alphanumeric chars.
*/
function normalize(s: string): string {
return s.toLowerCase().replace(/[-_.\/]/g, "");
}
/**
* Check if `needle` is a subsequence of `haystack` — every character of
* `needle` appears in `haystack` in the same order, but not necessarily
* contiguously.
*
* Both inputs should be pre-normalized.
*/
function isSubsequence(needle: string, haystack: string): boolean {
let ni = 0;
for (let hi = 0; hi < haystack.length && ni < needle.length; hi++) {
if (needle[ni] === haystack[hi]) ni++;
}
return ni === needle.length;
}
/**
* Check whether `needle` has edit distance ≤ `maxDist` to any contiguous
* substring of `haystack`, using a DP-based fuzzy-substring algorithm with
* Damerau-Levenshtein support (insertion, deletion, substitution, and
* adjacent transposition).
*
* The first DP row is initialised to zero so matching can begin at any
* position in `haystack` without penalty.
*/
function fuzzySubstring(needle: string, haystack: string, maxDist: number): boolean {
const n = needle.length;
const m = haystack.length;
if (n === 0) return true;
if (m === 0) return false;
// Two-rows-back buffer (needed for the transposition case)
let prev2 = new Array<number>(m + 1).fill(0);
// Previous DP row — initialised to 0 (free start position)
let prev = new Array<number>(m + 1).fill(0);
for (let i = 1; i <= n; i++) {
const curr = new Array<number>(m + 1);
curr[0] = i;
for (let j = 1; j <= m; j++) {
const cost = needle[i - 1] === haystack[j - 1] ? 0 : 1;
curr[j] = Math.min(
prev[j] + 1, // deletion
curr[j - 1] + 1, // insertion
prev[j - 1] + cost, // substitution
);
// Adjacent transposition (Damerau-Levenshtein)
if (
i >= 2 &&
j >= 2 &&
needle[i - 1] === haystack[j - 2] &&
needle[i - 2] === haystack[j - 1]
) {
curr[j] = Math.min(curr[j], prev2[j - 2] + 1);
}
}
prev2 = prev;
prev = curr;
}
// If any ending position has distance ≤ maxDist the needle fuzzy-matches
for (let j = 0; j <= m; j++) {
if (prev[j] <= maxDist) return true;
}
return false;
}
/**
* Check if a single search term matches the combined provider/id/name text,
* using these deterministic rules (any one is sufficient):
*
* 1. **Normalized substring** — separator-stripped, case-insensitive
* substring match against the full haystack (preserves original behaviour).
* 2. **Normalized subsequence** — characters appear in order within a single
* token (space-separated word) but need not be contiguous
* (e.g. `"cld"` → `"claude"`). Requires ≥ 3 chars to avoid short-query
* false positives.
* 3. **Typo tolerance** — Damerau-Levenshtein edit distance ≤ 1 to any
* substring of a single token, supporting single-char insertion, deletion,
* substitution, and adjacent transposition (e.g. `"sonet"` → `"sonnet"`).
* Requires ≥ 4 chars.
*
* Separators (hyphen, underscore, dot, slash) are stripped from *both*
* the term and the haystack before matching so that `"gpt4o"` matches
* `"gpt-4o"`.
*
* Subsequence and typo checks are scoped to individual tokens (space-split
* parts of the haystack) to prevent cross-field false positives.
*/
function termMatches(term: string, haystack: string): boolean {
const normTerm = normalize(term);
const normHaystack = normalize(haystack);
if (normTerm.length === 0) return true;
// 1. Substring match against full normalized haystack (any length)
if (normHaystack.includes(normTerm)) return true;
// For subsequence and fuzzy matching, check against individual tokens
// to avoid false positives from cross-field character picking.
const tokens = normHaystack.split(/\s+/).filter(Boolean);
for (const token of tokens) {
// 2. Subsequence match (min 3 chars to avoid short-query false positives)
if (normTerm.length >= 3 && isSubsequence(normTerm, token)) return true;
// 3. Typo-tolerant fuzzy match (min 4 chars)
if (normTerm.length >= 4 && fuzzySubstring(normTerm, token, 1)) return true;
}
return false;
}
/**
* Filter models by search terms matching provider, id, or name.
* Supports multi-word filters (space-separated AND logic).
* Case-insensitive substring matching.
*
* Supports multi-word filters with space-separated **AND** logic — every
* term must match independently. Matching is fuzzy: separator-insensitive,
* typo-tolerant (edit distance ≤ 1), and subsequence-aware, while still
* producing the same results as the old substring matcher for exact and
* substring queries.
*
* Result ordering is **stable** — models are returned in input-array order
* with no fuzzy-score re-sorting.
*/
export function filterModels(models: ModelInfo[], filter: string): ModelInfo[] {
const terms = filter.toLowerCase().trim().split(/\s+/).filter(Boolean);
if (terms.length === 0) return models;
return models.filter((m) => {
const haystack = `${m.provider} ${m.id} ${m.name}`.toLowerCase();
return terms.every((term) => haystack.includes(term));
const haystack = `${m.provider} ${m.id} ${m.name}`;
return terms.every((term) => termMatches(term, haystack));
});
}