8 Commits

Author SHA1 Message Date
Semih
1d5daf2e5c feat(insights): detect parts/catalog coverage failures (client + server)
Surfaces "vehicle resolved but no parts/categories" — Sase's #1 churn
signal, previously invisible to insights (a prospect emailed support
instead of any insight firing).

Client (PostHog session tagger):
- fetch the catalog_*/category_view_changed/empty_catalog_cta_clicked
  event family (was never pulled) and tag parts_render_blocked (P1) /
  catalog_empty_result (P2), scoped to the real /dashboard product (demo
  excluded). Harden the search-affordance-misuse guard so a real parts
  failure is never suppressed. Route both tags to bug_triage.

Server (decode-log catalog gaps):
- new catalog-gap-detect worker job (every 6h) -> panel internal API
  /api/internal/catalog-gap-check aggregates "No catalog - identified as
  X" query_logs failures by brand -> upserts catalog_coverage_gap
  insights (Telegram on P1), respecting founder triage. Add NO_CATALOG
  error bucket.

Validated: both apps typecheck clean; tagger smoke 21/21; aggregation
over 90d real data yields 13 brand gaps (Renault/Fiat/Honda P1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 14:22:09 +03:00
Semih
7a56a72037 fix(worker): graceful BullMQ worker shutdown to stop Missing-lock churn on deploy
On SIGTERM/SIGINT the worker only disconnected prisma/redis and exited — the
BullMQ Workers were never closed, so in-flight job locks were never released. On
every deploy the orchestrator killed the worker mid-flight, the next container
saw the half-finished jobs as "stalled", re-ran them, and the dead worker's
pending moveToFinished surfaced as `Missing lock for job
repeat:nightly-refresh:… moveToFinished`, with duplicate nightly-refresh firings
during the container overlap.

- startScheduledJobs / startInsightPipeline / startContentPipeline now return
  their Worker so index.ts can close them.
- shutdown() closes all workers FIRST (releases locks, drains in-flight),
  bounded by a 15s race so an in-flight job can't block past the orchestrator's
  stop grace period, then disconnects prisma/redis.
- nightly worker now uses lockDuration 5min + stalledInterval 60s (matching the
  pipeline/content workers); the 30s default could expire during panel-backup
  (pg_dump) and trip the same stalled → re-run → Missing-lock cycle.

The scheduler config itself was already correct (nightly@03:00, audit@03:30,
panel-backup@04:00 — verified against the Redis job-scheduler ZSET; next fires
were exactly 03:00/03:30/04:00). tsc --noEmit clean; dedup + tagger smoke 30/30.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 21:51:53 +03:00
Semih
556cfd6ba0 fix(insights): suppress no-insight search-affordance misuse in tagger
Users routinely use the VIN-only /dashboard/search box to look for a part by
name (e.g. "Cam düğme") or fiddle with search/history and rage-click out of
affordance confusion — while nothing is actually broken. The generic
ux_friction / frustrated_session tags turned these pure-rage sessions into
insights (noise; ~40 of the first 67 open insights were exactly this, all
dismissed by the founder).

apps/worker/src/lib/tagger.ts: when a session used the search box
(search_input_focused) but carries NO concrete failure (errorCount=0,
network5xxCount=0, no vin_decode_failed, search_input_validation_failed<3, no
payment_initiated/failed, no checkout_started), do not emit the generic
ux_friction / frustrated_session tags. With no other actionable tag the session
becomes tag-less and tag-sessions discards it → no compress / analyze / insight.

Scoped narrowly to search-box sessions on purpose, to avoid hiding genuine
parts/category bugs. Every concrete signal stays actionable: JS errors
(bug_suspected), 5xx (server_error_impact), payment friction, VIN upstream
provider failures, and ≥3 client validation failures (search_validation_friction).

Smoke: apps/worker/src/lib/tagger.smoke.ts (12/12) covers the misuse case plus
six must-keep cases. tsc --noEmit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 21:35:58 +03:00
Semih
b61d955256 feat(insights): semantic dedup gate + dismissed-theme suppression in analyze
The fingerprint hash (even after a7fe80f) still over-splits: the same root
problem produces different tag-sets / first-error / first-failed-endpoint
across sessions, so it hashes to a new fingerprint and creates a duplicate
insight. And generic ux_friction/frustrated_session sessions (category-tree
vs parts-panel vs schema-image) carry no structured signal to separate
sub-problems — only the LLM's reading of the timeline does. Dismissed insights
were only suppressed on an EXACT fingerprint repeat; a near-variant got a fresh
fingerprint and re-opened the theme. Dismissed rows were also excluded from the
fast-path cache, so every recurring session re-ran the LLM for no reason.

Two-layer dedup so (1) duplicates can't be created and (2) nothing similar to a
dismissed/duplicate theme is re-opened:

Layer 1 — fingerprint (or aliased fingerprint) fast-path, no LLM:
  - dismissed/duplicate  → SUPPRESS (bump occ/lastSeen, keep status)
  - validated/shipped    → flag REGRESSION (cheap, no re-analysis)
  - active & fresh        → attach occurrence
  - active & stale        → fall through to refresh re-analysis (unchanged)

Layer 2 — semantic dedup gate before creating a NEW insight (flash, no new infra):
  - build a bounded catalog: all active insights + recently-touched
    dismissed/duplicate/validated/shipped (lookback-windowed, capped)
  - ask the model whether the session is the SAME underlying problem as a
    catalog entry (anti-hallucination: only accept catalog ids; conf >= 0.72)
  - on match: alias this fingerprint onto that insight + attach/suppress, so the
    next identical session fast-paths in Layer 1 (no LLM, no duplicate row)
  - no match: create a new insight as before

Schema: Insight.aliasFingerprints String[] (additive; applied via prisma db push
on deploy). Dedup-gate cost is logged to costLedger as promptTag=dedup_gate.

Tunables: INSIGHT_DEDUP_GATE (default on), INSIGHT_DEDUP_THRESHOLD (0.72),
INSIGHT_DEDUP_SUPPRESS_DAYS (90), INSIGHT_DEDUP_CATALOG_MAX (120).

Pure decision helpers (classifyExistingAction, shouldAcceptMatch) extracted and
covered by a smoke test (apps/worker `pnpm test`, 18/18). tsc --noEmit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 20:41:18 +03:00
Semih
a7fe80f0c5 fix(insights): tighten session fingerprint so duplicate UX issues dedupe
Same root cause was producing one Insight row per user because URL carried
vehicleId/categoryId UUIDs and the raw URL fed the fingerprint hash. Result:
6 active "new" insights describing the same kategori-bouncing problem with
slightly different LLM phrasing, none deduplicated, only one (cmpclg7b9 →
sase.tr#76) had been triaged.

apps/worker/src/lib/compress.ts:
- new normalizePath() — collapses UUID / ULID / CUID2 / numeric path segments
  to ":id", conservative on plain words. Mirrors what the LLM already sees in
  the timeline.
- fingerprint inputs now:
    tags (sorted) | normalizePath(url) | normalizeError(errors[0]) | normalizePath(failedEndpoints[0])
- removed header.severity from the hash — severity is a property of the
  Insight bucket, not its identity; rage-click counts pushing the same root
  cause across P1/P2/P3 was forcing extra rows.

Smoke (12/12 pass via tsx /tmp/check_fingerprint.ts):
- 6 historical kategori-bouncing URLs → 1 fingerprint
- severity changes don't move the hash
- distinct tag sets / distinct errors still split

Historical rows are untouched — only new compressed sessions get the new
hash. Old near-duplicate insights can be merged manually via the panel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 19:42:25 +03:00
Semih
9a479f9a5b fix(insights): distinguish client-side VIN validation rejects from provider failures
Reproducing insight cmpvfrjgc000114fzc7cdyh66 (a P1 "PL24 timeout" false
positive): trial user typed VW part numbers ("500 907 521", "5Q0 907 521")
into the VIN field on /; client-side regex rejected them with "Geçersiz şase
numarası. 17 karakter olmalı". No provider was called. The pipeline still
tagged the session as `vin_decode_fail_pattern`, routed to `provider_quality`,
and the LLM dutifully invented a PL24 outage.

Root cause spans three files:

1. tagger.ts grouped vin_decode_failed by `provider_attempted ?? source`. When
   `provider_attempted` is missing, `source: "landing"` (a UI location) was
   treated as a provider name, so a 1-provider set was synthesized and
   `vin_decode_fail_pattern` (P1) was emitted.

2. compress.ts formatCustom whitelist excluded `error`, `source`, `vin`. The
   LLM therefore never saw "Geçersiz şase numarası" or the offending input.
   Pattern 3 mechanical hypothesis told it "check provider health" regardless.

3. prompts.ts pickPromptTag routed any `vin_decode_fail_pattern` straight to
   `provider_quality` with no input-quality check, and the v3 system prompt
   had no guardrail for client-side validation rejects.

Fix:
- tagger: detect client-side rejects by `error` regex (Turkish + English) and
  by VIN shape (length != 17 or contains I/O/Q). When all fails are client
  rejects, emit new tag `vin_decode_client_validation_fail` at P3 instead of
  `vin_decode_fail_pattern` at P1. Real provider failures now require
  `provider_attempted` to be set (no more `source` fallback).
- compress: add `error`, `source`, `vin` to the formatCustom property
  whitelist so the LLM can see the actual failure context. Split Pattern 3
  into client-reject vs. real-provider-failure branches with distinct
  Turkish hypotheses.
- prompts: route `vin_decode_client_validation_fail` to `ux_friction` before
  the provider rule. Ship provider_quality v4 with an explicit guardrail
  instructing the model to return confidence ≤0.15 and reclassify when the
  inlined event properties show client-side rejection.

The seed-runtime upsert path deactivates the active v3 template on next
worker boot and inserts v4 in its place — no manual SQL needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 23:12:39 +03:00
59cb1f63ec Merge pull request 'feat(insights): multi-project Sentry archive' (#3) from feat/sentry-multi-project into main 2026-06-01 15:38:16 +00:00
7eca679ab8 Merge pull request 'feat(insights): permanent archive of PostHog (events+recordings+identity) + Sentry (Phase A+B+C)' (#2) from feat/observability-archive into main 2026-05-27 19:13:31 +00:00
20 changed files with 1234 additions and 69 deletions

View File

@@ -251,6 +251,11 @@ model Insight {
severity String severity String
status String @default("new") status String @default("new")
fingerprint String fingerprint String
// Additional fingerprints semantically merged into this insight by the dedup
// gate: a new session whose own fingerprint differs but which the LLM judged
// to be the SAME underlying problem gets its fingerprint aliased here, so the
// next identical session fast-paths (no re-analysis, no duplicate row).
aliasFingerprints String[] @default([])
title String title String
body Json body Json
relatedSessionIds String[] relatedSessionIds String[]

View File

@@ -0,0 +1,29 @@
import { NextResponse } from "next/server";
import { timingSafeEqual } from "node:crypto";
import { detectCatalogCoverageGaps } from "@/lib/sase/catalog-coverage";
export const dynamic = "force-dynamic";
const INTERNAL_TOKEN = process.env.INTERNAL_WORKER_TOKEN ?? "";
function validToken(req: Request): boolean {
if (!INTERNAL_TOKEN) return false;
const provided = req.headers.get("x-internal-worker-token") ?? "";
if (!provided) return false;
const a = Buffer.from(provided);
const b = Buffer.from(INTERNAL_TOKEN);
if (a.length !== b.length) return false;
return timingSafeEqual(a, b);
}
export async function GET(req: Request) {
if (!validToken(req)) {
return NextResponse.json({ ok: false, error: "unauthorized" }, { status: 401 });
}
try {
const gaps = await detectCatalogCoverageGaps({ windowDays: 7, minFailures: 3 });
return NextResponse.json({ ok: true, gaps });
} catch (e) {
return NextResponse.json({ ok: false, error: (e as Error).message }, { status: 500 });
}
}

View File

@@ -0,0 +1,77 @@
import { saseDb } from "@/lib/db-sase";
// ─── Catalog coverage gaps ────────────────────────────────────────────────
// The single biggest VIN-decode failure mode is "No catalog — identified as X":
// the decoder *recognises* the vehicle (brand/year) but Sase has no parts
// catalog mapped for it, so the user sees their car but no parts. These rows
// land in query_logs with success=false and error_message starting
// "No catalog — identified as <Brand> [<Year>]". We aggregate them by brand so
// the worker can raise ONE insight per brand-gap instead of per failed query.
export type CatalogGap = {
brand: string;
failures: number;
uniqueUsers: number;
labels: string[]; // distinct "<Brand> <Year>" identities seen, e.g. ["Renault 2004","Renault 2006"]
years: number[]; // parsed model years (sorted)
firstSeen: string; // ISO
lastSeen: string; // ISO
windowDays: number;
};
export async function detectCatalogCoverageGaps(
opts: { windowDays?: number; minFailures?: number } = {},
): Promise<CatalogGap[]> {
const windowDays = opts.windowDays ?? 7;
const minFailures = opts.minFailures ?? 3;
// Brand is the first token of the identified label; initcap() folds HONDA/Honda
// into one group. array_agg collects the distinct "<Brand> <Year>" identities so
// the worker can show the affected model years.
const rows = await saseDb.$queryRaw<
Array<{
brand: string;
failures: number;
unique_users: number;
first_seen: Date;
last_seen: Date;
labels: string[];
}>
>`
WITH nc AS (
SELECT user_id, created_at,
trim(substring(error_message FROM 'identified as (.*)$')) AS label
FROM query_logs
WHERE success = false
AND created_at > now() - make_interval(days => ${windowDays})
AND error_message LIKE 'No catalog%'
)
SELECT
initcap(split_part(label, ' ', 1)) AS brand,
count(*)::int AS failures,
count(distinct user_id)::int AS unique_users,
min(created_at) AS first_seen,
max(created_at) AS last_seen,
array_agg(DISTINCT label ORDER BY label) AS labels
FROM nc
WHERE label IS NOT NULL AND label <> ''
GROUP BY 1
HAVING count(*) >= ${minFailures}
ORDER BY failures DESC
`;
return rows.map((r) => {
const years = Array.from(
new Set(r.labels.flatMap((l) => (l.match(/\b(?:19|20)\d{2}\b/g) ?? []).map(Number))),
).sort((a, b) => a - b);
return {
brand: r.brand,
failures: r.failures,
uniqueUsers: r.unique_users,
labels: r.labels,
years,
firstSeen: r.first_seen.toISOString(),
lastSeen: r.last_seen.toISOString(),
windowDays,
};
});
}

View File

@@ -260,6 +260,7 @@ export type ErrorBucketRow = {
export const ERROR_BUCKET_KEYS = [ export const ERROR_BUCKET_KEYS = [
"BUDGET_EXCEEDED", "BUDGET_EXCEEDED",
"NO_CATALOG",
"UNKNOWN_VIN", "UNKNOWN_VIN",
"TIMEOUT", "TIMEOUT",
"INVALID_VIN", "INVALID_VIN",
@@ -273,6 +274,9 @@ export type ErrorBucketKey = (typeof ERROR_BUCKET_KEYS)[number];
const ERROR_PATTERNS: Array<{ key: ErrorBucketKey; matchers: RegExp[] }> = [ const ERROR_PATTERNS: Array<{ key: ErrorBucketKey; matchers: RegExp[] }> = [
{ key: "BUDGET_EXCEEDED", matchers: [/budget/i, /aborted/i] }, { key: "BUDGET_EXCEEDED", matchers: [/budget/i, /aborted/i] },
// Vehicle identified but no parts catalog mapped — the #1 "OTHER" failure and
// the server-side twin of the catalog_empty_result insight signal.
{ key: "NO_CATALOG", matchers: [/no catalog/i] },
{ key: "UNKNOWN_VIN", matchers: [/unknown vin/i, /tanınamadı/i, /destekl/i] }, { key: "UNKNOWN_VIN", matchers: [/unknown vin/i, /tanınamadı/i, /destekl/i] },
{ key: "TIMEOUT", matchers: [/timeout/i, /timed out/i] }, { key: "TIMEOUT", matchers: [/timeout/i, /timed out/i] },
{ key: "INVALID_VIN", matchers: [/geçersiz/i, /invalid vin/i] }, { key: "INVALID_VIN", matchers: [/geçersiz/i, /invalid vin/i] },

View File

@@ -5,7 +5,8 @@
"scripts": { "scripts": {
"dev": "tsx watch src/index.ts", "dev": "tsx watch src/index.ts",
"start": "tsx src/index.ts", "start": "tsx src/index.ts",
"typecheck": "tsc --noEmit" "typecheck": "tsc --noEmit",
"test": "tsx src/lib/dedup.smoke.ts && tsx src/lib/tagger.smoke.ts"
}, },
"dependencies": { "dependencies": {
"@panel/web": "workspace:*", "@panel/web": "workspace:*",

View File

@@ -1,3 +1,4 @@
import type { Worker } from "bullmq";
import { startEventBus } from "./consumers/event-bus"; import { startEventBus } from "./consumers/event-bus";
import { startScheduledJobs } from "./schedulers/nightly"; import { startScheduledJobs } from "./schedulers/nightly";
import { startInsightPipeline } from "./schedulers/pipeline"; import { startInsightPipeline } from "./schedulers/pipeline";
@@ -6,6 +7,8 @@ import { upsertSeedData } from "./lib/seed-runtime";
import { redis } from "./redis"; import { redis } from "./redis";
import { prisma } from "./db"; import { prisma } from "./db";
const workers: Worker[] = [];
async function main() { async function main() {
console.log("[worker] starting…"); console.log("[worker] starting…");
await redis.ping(); await redis.ping();
@@ -15,16 +18,27 @@ async function main() {
await upsertSeedData().catch((e) => console.warn("[seed] failed:", e.message)); await upsertSeedData().catch((e) => console.warn("[seed] failed:", e.message));
await startScheduledJobs(); workers.push(await startScheduledJobs());
await startInsightPipeline(); workers.push(await startInsightPipeline());
await startContentPipeline(); workers.push(await startContentPipeline());
await startEventBus(); await startEventBus();
console.log("[worker] up."); console.log("[worker] up.");
} }
let shuttingDown = false;
const shutdown = async (sig: string) => { const shutdown = async (sig: string) => {
console.log(`[worker] ${sig}shutting down`); if (shuttingDown) return;
shuttingDown = true;
console.log(`[worker] ${sig} — closing ${workers.length} workers gracefully`);
// Close workers FIRST: this stops them taking new jobs and releases held job
// locks, so the next container doesn't inherit half-finished jobs as "stalled"
// and spam "Missing lock … moveToFinished". Bounded so an in-flight job can't
// block the shutdown past the orchestrator's stop grace period.
await Promise.race([
Promise.allSettled(workers.map((w) => w.close())),
new Promise((resolve) => setTimeout(resolve, 15_000)),
]);
await prisma.$disconnect().catch(() => {}); await prisma.$disconnect().catch(() => {});
await redis.quit().catch(() => {}); await redis.quit().catch(() => {});
process.exit(0); process.exit(0);

View File

@@ -1,10 +1,18 @@
import { prisma } from "../db"; import { prisma } from "../db";
import { callDeepSeek, extractJson, type Tier, DeepSeekError } from "../lib/deepseek"; import { callDeepSeek, extractJson, type Tier, type CallResult, DeepSeekError } from "../lib/deepseek";
import { checkBudget } from "../lib/budget"; import { checkBudget } from "../lib/budget";
import { pickPromptTag } from "../lib/prompts"; import { pickPromptTag } from "../lib/prompts";
import { validate } from "../lib/json-validate"; import { validate } from "../lib/json-validate";
import { getText } from "../lib/minio"; import { getText } from "../lib/minio";
import { alertP0Insight, alertBudgetCap } from "../lib/telegram"; import { alertP0Insight, alertBudgetCap } from "../lib/telegram";
import {
buildInsightCatalog,
findSemanticMatch,
classifyExistingAction,
shouldAcceptMatch,
DEDUP_ENABLED,
DEDUP_THRESHOLD,
} from "../lib/dedup";
const PANEL_URL = process.env.PANEL_PUBLIC_URL ?? "https://sp.semih.ai"; const PANEL_URL = process.env.PANEL_PUBLIC_URL ?? "https://sp.semih.ai";
@@ -22,6 +30,88 @@ export type AnalyzeResult = {
budgetState: string; budgetState: string;
}; };
type GroupSession = { id: string; startedAt: Date };
type ExistingInsight = {
id: string;
status: string;
relatedSessionIds: string[];
lastSeenAt: Date;
updatedAt: Date;
aliasFingerprints: string[];
};
async function markAnalyzed(ids: string[]): Promise<void> {
await prisma.sessionMeta.updateMany({
where: { id: { in: ids } },
data: { status: "analyzed", processedAt: new Date() },
});
}
/**
* Attach a session group to an existing insight WITHOUT re-running the LLM.
* Returns true when fully handled (caller should `continue`). Returns false
* only for an active-but-stale insight when the caller allows a refresh
* re-analysis (Layer 1, same fingerprint) — the caller then falls through to
* the normal analysis/update path.
*
* - dismissed / duplicate → suppress silently (respect the triage decision)
* - validated / shipped → flag regression (cheap, no re-analysis)
* - active & recent → attach occurrence
*/
async function attachToExisting(
existing: ExistingInsight,
group: GroupSession[],
opts: { allowStaleReanalyze: boolean; cutoff: Date },
): Promise<boolean> {
const action = classifyExistingAction(
existing.status,
existing.updatedAt <= opts.cutoff,
opts.allowStaleReanalyze,
);
if (action === "reanalyze") return false; // active & stale → caller refreshes
const rel = Array.from(new Set([...existing.relatedSessionIds, ...group.map((g) => g.id)]));
const groupLast = group.reduce((a, b) => (a.startedAt > b.startedAt ? a : b)).startedAt;
const lastSeenAt = groupLast > existing.lastSeenAt ? groupLast : existing.lastSeenAt;
await prisma.insight.update({
where: { id: existing.id },
data: {
relatedSessionIds: rel,
occurrenceCount: rel.length,
lastSeenAt,
...(action === "regress" ? { status: "regressed", regressionDetected: true } : {}),
},
});
await markAnalyzed(group.map((g) => g.id));
return true;
}
async function logDedupCost(sessionId: string, projectKey: string, raw: CallResult): Promise<void> {
await prisma.costLedger.create({
data: {
sessionId,
projectKey,
promptTag: "dedup_gate",
promptVersion: 0,
provider: "deepseek",
model: raw.model,
tier: "flash",
tokensInputCacheMiss: raw.usage.inputTokensMiss,
tokensInputCacheHit: raw.usage.inputTokensHit,
tokensOutput: raw.usage.outputTokens,
costInputCacheMissUsd: raw.cost.inputMissUsd,
costInputCacheHitUsd: raw.cost.inputHitUsd,
costOutputUsd: raw.cost.outputUsd,
costTotalUsd: raw.cost.totalUsd,
cacheHitRatio: raw.cost.cacheHitRatio,
callDurationMs: raw.durationMs,
errorCode: null,
},
});
}
export async function runAnalyze(): Promise<AnalyzeResult> { export async function runAnalyze(): Promise<AnalyzeResult> {
const budget = await checkBudget(); const budget = await checkBudget();
if (!budget.allow) { if (!budget.allow) {
@@ -72,31 +162,73 @@ export async function runAnalyze(): Promise<AnalyzeResult> {
continue; continue;
} }
// Application-level fingerprint cache: if an active insight with same fingerprint // ─── Layer 1: exact fingerprint (or aliased fingerprint) fast-path ───
// exists and is younger than CACHE_TTL_HOURS, just attach this session to it. // A session whose fingerprint already maps to an insight is attached without
// re-running the LLM: dismissed/duplicate SUPPRESS it (respect triage — no
// new row, no cost), validated/shipped FLAG a regression, active+recent just
// accumulate. Only active+stale falls through to a refresh re-analysis.
const cutoff = new Date(Date.now() - CACHE_TTL_HOURS * 3600_000); const cutoff = new Date(Date.now() - CACHE_TTL_HOURS * 3600_000);
const existing = await prisma.insight.findUnique({ const existing = await prisma.insight.findFirst({
where: { projectKey_fingerprint: { projectKey: PROJECT_KEY, fingerprint: s.fingerprint } }, where: {
projectKey: PROJECT_KEY,
OR: [{ fingerprint: s.fingerprint }, { aliasFingerprints: { has: s.fingerprint } }],
},
}); });
if (existing && existing.updatedAt > cutoff && !["dismissed", "validated"].includes(existing.status)) { if (existing) {
// Aggregate this session into the existing insight const handled = await attachToExisting(existing, group, { allowStaleReanalyze: true, cutoff });
const rel = Array.from(new Set([...existing.relatedSessionIds, s.id])); if (handled) {
await prisma.insight.update({ skipped++;
where: { id: existing.id }, continue;
data: { }
relatedSessionIds: rel, // active & stale → fall through to full re-analysis (existing stays set).
occurrenceCount: rel.length, }
lastSeenAt: s.startedAt > existing.lastSeenAt ? s.startedAt : existing.lastSeenAt,
}, // Fetch timeline from MinIO — needed by both the dedup gate and the analysis.
}); let timeline: string;
await prisma.sessionMeta.update({ try {
where: { id: s.id }, timeline = await getText(COMPRESSION_BUCKET, s.compressed.semanticTimelineMinioKey);
data: { status: "analyzed", processedAt: new Date() }, } catch (e) {
}); console.warn(`[analyze] timeline fetch failed ${s.id}: ${(e as Error).message}`);
skipped++; failed++;
continue; continue;
} }
// ─── Layer 2: semantic dedup gate (only for genuinely new fingerprints) ───
// Before spending the expensive analysis call, ask the flash model whether
// this session is the SAME underlying problem as an existing insight —
// including dismissed/duplicate ones we must NOT recreate. On a confident
// match we alias this fingerprint onto that insight and attach/suppress, so
// future identical sessions fast-path with no LLM and no duplicate row.
if (!existing && DEDUP_ENABLED) {
const catalog = await buildInsightCatalog(PROJECT_KEY);
const match = await findSemanticMatch({ timeline, tags: s.tags, catalog });
if (match.raw) {
costUsd += match.raw.cost.totalUsd;
await logDedupCost(s.id, PROJECT_KEY, match.raw);
}
const catalogIds = new Set(catalog.map((c) => c.id));
if (match.matchId && shouldAcceptMatch(match.matchId, match.confidence, catalogIds, DEDUP_THRESHOLD)) {
const matchId = match.matchId;
const matched = await prisma.insight.findUnique({ where: { id: matchId } });
if (matched) {
await prisma.insight.update({
where: { id: matched.id },
data: {
aliasFingerprints: Array.from(new Set([...matched.aliasFingerprints, s.fingerprint])),
},
});
await attachToExisting(matched, group, { allowStaleReanalyze: false, cutoff });
console.log(
`[analyze] dedup-gate matched ${s.fingerprint}${matched.id} ` +
`(${matched.status}) conf=${match.confidence.toFixed(2)} :: ${match.reason}`,
);
skipped++;
continue;
}
}
// no confident match → fall through and create a new insight below.
}
const promptTag = isBundle ? "pattern_bundle" : pickPromptTag(s.tags); const promptTag = isBundle ? "pattern_bundle" : pickPromptTag(s.tags);
const template = await prisma.promptTemplate.findFirst({ const template = await prisma.promptTemplate.findFirst({
where: { tag: promptTag, active: true }, where: { tag: promptTag, active: true },
@@ -114,16 +246,6 @@ export async function runAnalyze(): Promise<AnalyzeResult> {
else if (s.severity === "P2" || s.severity === "P3" || s.severity === "INFO") tier = "flash"; else if (s.severity === "P2" || s.severity === "P3" || s.severity === "INFO") tier = "flash";
if (budget.forceTier) tier = budget.forceTier; if (budget.forceTier) tier = budget.forceTier;
// Fetch timeline from MinIO (primary)
let timeline: string;
try {
timeline = await getText(COMPRESSION_BUCKET, s.compressed.semanticTimelineMinioKey);
} catch (e) {
console.warn(`[analyze] timeline fetch failed ${s.id}: ${(e as Error).message}`);
failed++;
continue;
}
// For bundles, append a summary of the other sessions in the group. // For bundles, append a summary of the other sessions in the group.
if (isBundle) { if (isBundle) {
const otherSessionIds = group.slice(1).map((x) => x.id); const otherSessionIds = group.slice(1).map((x) => x.id);

View File

@@ -0,0 +1,196 @@
import { Prisma } from "@prisma/client";
import { prisma } from "../db";
import { alertCatalogGap, isTelegramConfigured } from "../lib/telegram";
// Server-side companion to the session-driven insight pipeline. The behavioural
// pipeline (PostHog → tagger → analyze) catches "user resolved a vehicle but saw
// no parts" from client events (catalog_empty_result / parts_render_blocked).
// This job catches the SAME failure from the server side — query_logs rows where
// decode succeeded at identifying the car but no parts catalog exists
// ("No catalog — identified as X") — and raises one Insight per brand-gap into
// the founder's existing inbox. Panel owns the Sase DB; we fetch the aggregate
// over the internal API (same trust model as vin-anomaly-detect) and write rows.
const PANEL_BASE =
process.env.PANEL_INTERNAL_URL ?? process.env.PANEL_PUBLIC_URL ?? "http://panel-web:3000";
const PANEL_PUBLIC = process.env.PANEL_PUBLIC_URL ?? "https://sp.semih.ai";
const WORKER_TOKEN = process.env.INTERNAL_WORKER_TOKEN ?? "";
const PROJECT_KEY = process.env.INSIGHT_PROJECT_KEY ?? "sase";
type CatalogGap = {
brand: string;
failures: number;
uniqueUsers: number;
labels: string[];
years: number[];
firstSeen: string;
lastSeen: string;
windowDays: number;
};
type CheckResponse = { ok: boolean; error?: string; gaps?: CatalogGap[] };
type Summary = {
ok: boolean;
gaps: number;
created: number;
updated: number;
alertsFired: number;
reason?: string;
};
function brandSlug(b: string): string {
return b.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/(^-|-$)/g, "");
}
// P1 once a gap bites real breadth (≥4 distinct users or ≥15 failed lookups in
// the window) — a core-value failure for a whole brand; otherwise P2.
function severityFor(g: CatalogGap): "P1" | "P2" {
return g.uniqueUsers >= 4 || g.failures >= 15 ? "P1" : "P2";
}
function priorityFor(sev: string, users: number): number {
return (sev === "P1" ? 80 : 55) + Math.min(15, users);
}
// Shape the body around keys the insight detail page already renders
// (hypothesis / affected_route / user_impact_estimate / suggested_investigation /
// reproduce_steps); extra keys still show in the Raw JSON panel.
function buildBody(g: CatalogGap): Prisma.InputJsonValue {
const yrs = g.years.length ? ` (${g.years[0]}${g.years[g.years.length - 1]})` : "";
const labelList = g.labels.slice(0, 12).join(", ");
return {
summary: `${g.brand}${yrs}: araç decode'da tanınıyor ama parça kataloğu bulunamıyor ("No catalog"). Son ${g.windowDays} günde ${g.failures} başarısız sorgu / ${g.uniqueUsers} kullanıcı.`,
hypothesis: `Decode aracı doğru tanıyor ama "tanınan araç → parça kataloğu" eşlemesi ${g.brand} için boş dönüyor. Hatalar belirli model yıllarında kümeleniyor (${labelList}) → muhtemelen katalog MAPPING eksikliği (EMEX/PCAT kaynağında veri var ama brand/yıl eşleşmiyor) ya da o segment için kaynak verisi hiç yok.`,
affected_route: `/dashboard/catalog/${g.brand}, /dashboard/vehicles/:id (decode → "No catalog")`,
user_impact_estimate: `${g.uniqueUsers} kullanıcı son ${g.windowDays} günde ${g.brand} için parça göremedi → doğrudan churn sinyali. ${g.brand} TR pazarında yaygın bir marka.`,
suggested_investigation: [
`vehicles.service'teki katalog lookup zincirinde "No catalog" branch'ini incele (apps/api .../vehicles/vehicles.service.ts)`,
`sase-catalog-src-emex / sase-catalog-src-pcat DB'lerinde ${g.brand} (${labelList}) var mı — veri mi eksik, mapping mi bozuk?`,
`Tanınan brand/model_year → catalog brand/subcatalog eşlemesini kontrol et`,
],
reproduce_steps: [
`Katalogdan ${g.brand} seç (ya da bu markaya ait bir VIN decode et)`,
`Bir model/yıl seç (${g.labels[0] ?? g.brand})`,
`Parça/kategori yerine boş sonuç / "No catalog" gözlenir`,
],
suggested_fix_effort: "M",
affected_labels: g.labels,
total_failures: g.failures,
unique_users: g.uniqueUsers,
window_days: g.windowDays,
};
}
export async function runCatalogGapDetect(): Promise<Summary> {
if (!WORKER_TOKEN) {
return { ok: false, gaps: 0, created: 0, updated: 0, alertsFired: 0, reason: "INTERNAL_WORKER_TOKEN not set" };
}
let res: Response;
try {
res = await fetch(`${PANEL_BASE}/api/internal/catalog-gap-check`, {
headers: { "x-internal-worker-token": WORKER_TOKEN, "cache-control": "no-store" },
});
} catch (e) {
return { ok: false, gaps: 0, created: 0, updated: 0, alertsFired: 0, reason: `fetch failed: ${(e as Error).message}` };
}
if (!res.ok) {
return { ok: false, gaps: 0, created: 0, updated: 0, alertsFired: 0, reason: `panel returned ${res.status}` };
}
const data = (await res.json()) as CheckResponse;
if (!data.ok) {
return { ok: false, gaps: 0, created: 0, updated: 0, alertsFired: 0, reason: data.error };
}
const gaps = data.gaps ?? [];
const day = new Date().toISOString().slice(0, 10);
let created = 0;
let updated = 0;
let alertsFired = 0;
for (const g of gaps) {
const fingerprint = `catalog_gap:${brandSlug(g.brand)}`;
const sev = severityFor(g);
const body = buildBody(g);
const title = `Katalog yok: ${g.brand}${g.uniqueUsers} kullanıcı parça göremiyor (son ${g.windowDays}g)`;
const existing = await prisma.insight.findUnique({
where: { projectKey_fingerprint: { projectKey: PROJECT_KEY, fingerprint } },
});
if (existing) {
// Refresh counts/severity but RESPECT founder triage: a dismissed/duplicate
// gap stays dismissed (no resurrection). A gap the founder had already
// validated/shipped that is failing again flips to "regressed" + alerts.
const regressed = ["validated", "shipped"].includes(existing.status);
await prisma.insight.update({
where: { id: existing.id },
data: {
title,
body,
severity: sev,
occurrenceCount: g.failures,
uniqueUserCount: g.uniqueUsers,
lastSeenAt: new Date(g.lastSeen),
priorityScore: priorityFor(sev, g.uniqueUsers),
...(regressed ? { status: "regressed", regressionDetected: true } : {}),
},
});
updated++;
if (regressed && isTelegramConfigured()) {
const r = await alertCatalogGap({
brand: g.brand,
failures: g.failures,
uniqueUsers: g.uniqueUsers,
windowDays: g.windowDays,
severity: sev,
insightId: existing.id,
panelUrl: PANEL_PUBLIC,
day,
regressed: true,
});
if (r.ok && !r.deduped) alertsFired++;
}
continue;
}
const row = await prisma.insight.create({
data: {
projectKey: PROJECT_KEY,
type: "catalog_coverage_gap",
severity: sev,
status: "new",
fingerprint,
title,
body,
relatedSessionIds: [],
occurrenceCount: g.failures,
uniqueUserCount: g.uniqueUsers,
firstSeenAt: new Date(g.firstSeen),
lastSeenAt: new Date(g.lastSeen),
confidence: 1.0,
priorityScore: priorityFor(sev, g.uniqueUsers),
sourcePromptTag: "catalog_gap_detector",
sourcePromptVersion: 0,
sourceModel: "rule:catalog-gap",
sourceCostUsd: 0,
},
});
created++;
if (isTelegramConfigured()) {
const r = await alertCatalogGap({
brand: g.brand,
failures: g.failures,
uniqueUsers: g.uniqueUsers,
windowDays: g.windowDays,
severity: sev,
insightId: row.id,
panelUrl: PANEL_PUBLIC,
day,
regressed: false,
});
if (r.ok && !r.deduped) alertsFired++;
}
}
return { ok: true, gaps: gaps.length, created, updated, alertsFired };
}

View File

@@ -145,11 +145,17 @@ export function compressSnapshots(
const formatCustom = (ev: CanonicalEvent): string => { const formatCustom = (ev: CanonicalEvent): string => {
// Inline a tiny subset of important properties to keep tokens bounded. // Inline a tiny subset of important properties to keep tokens bounded.
// `error`, `source`, `vin` are essential for vin_decode_failed disambiguation
// (client-side validation reject vs. upstream provider failure) — without
// them the LLM cannot tell whether a failure actually reached a provider.
const p = ev.properties ?? {}; const p = ev.properties ?? {};
const keys = [ const keys = [
"provider", "provider",
"provider_attempted", "provider_attempted",
"error_code", "error_code",
"error",
"source",
"vin",
"result", "result",
"plan", "plan",
"amount", "amount",
@@ -391,11 +397,39 @@ export function compressSnapshots(
hypotheses.push("Fresh trial activated and decoded a VIN successfully — onboarding succeeded"); hypotheses.push("Fresh trial activated and decoded a VIN successfully — onboarding succeeded");
} }
// Pattern 3: VIN decode failure or provider fallback — investigate upstream. // Pattern 3: VIN decode failure or provider fallback — but separate the
const vinFailures = count("vin_decode_failed") + count("vin_decode_error"); // client-side validation rejects (input never reached a provider) from real
if (vinFailures >= 2 || count("provider_fallback_triggered") >= 1) { // upstream failures. Lumping them together biases the LLM toward "provider
// issue" verdicts when the actual signal is user input affordance.
const vinFailEvents = customEvents.filter(
(c) => c.name === "vin_decode_failed" || c.name === "vin_decode_error",
);
const clientRejects = vinFailEvents.filter((c) => {
const p = (c.properties ?? {}) as Record<string, unknown>;
const err = String(p.error ?? "");
const vin = String(p.vin ?? "");
if (/(Geçersiz şase|17 karakter|I, O, Q|invalid VIN|must be 17|format)/i.test(err)) return true;
if (vin && vin.replace(/\s/g, "").length !== 17) return true;
if (vin && /[IOQ]/i.test(vin)) return true;
return false;
});
const realProviderFails = vinFailEvents.length - clientRejects.length;
const fallbacks = count("provider_fallback_triggered");
if (clientRejects.length >= 1 && realProviderFails === 0) {
const sample = String(
((clientRejects[0].properties ?? {}) as Record<string, unknown>).vin ?? "",
).slice(0, 24);
hypotheses.push( hypotheses.push(
`VIN decode failure pattern (failures=${vinFailures}, fallbacks=${count("provider_fallback_triggered")}) — check provider health`, `VIN inputuna geçersiz format girildi (${clientRejects.length}x client-side validation reddi${sample ? `, örn. "${sample}"` : ""}) — provider çağrılmadı, input affordance / yanlış alan kullanımı problemi`,
);
} else if (realProviderFails >= 2 || fallbacks >= 1) {
hypotheses.push(
`VIN decode failure pattern (provider failures=${realProviderFails}, fallbacks=${fallbacks}) — check provider health`,
);
} else if (realProviderFails === 1) {
hypotheses.push(
`Single VIN decode failure reached a provider — likely transient, watch for repeat`,
); );
} }
@@ -440,12 +474,23 @@ export function compressSnapshots(
...(hypotheses.length ? hypotheses.map((h) => `- ${h}`) : ["- (none)"]), ...(hypotheses.length ? hypotheses.map((h) => `- ${h}`) : ["- (none)"]),
].join("\n"); ].join("\n");
// Fingerprint inputs are intentionally coarse so that recurring problems
// cluster into a single Insight row instead of producing a new row per
// user/vehicle/category. Two prior issues this guards against:
// 1. URL carries vehicleId/categoryId UUIDs — every user produces a unique
// URL, so identical kategori-bouncing sessions never deduplicate.
// normalizePath collapses UUIDs/ULIDs/CUIDs/numeric ids to ":id".
// 2. header.severity is derived from rage-click count and varies between
// P1/P2/P3 for the same root cause — it should be a property of the
// Insight, not part of its identity. Dropped from the hash.
// Tag set + normalized path + first-error + first-failed-endpoint give
// enough discrimination because distinct UX failures already carry distinct
// tagger tags (vin_decode_*, payment_*, search_validation_*, etc.).
const fingerprint = fingerprintHash([ const fingerprint = fingerprintHash([
[...header.tags].sort().join(","), [...header.tags].sort().join(","),
url, normalizePath(url),
errors[0] ? normalizeError(errors[0]) : null, errors[0] ? normalizeError(errors[0]) : null,
failedEndpoints[0] ?? null, failedEndpoints[0] ? normalizePath(failedEndpoints[0]) : null,
header.severity,
]); ]);
return { return {
@@ -477,6 +522,47 @@ function stripQuery(u: string): string {
} }
} }
/**
* Collapse ID-like path segments to `:id` so URLs with embedded
* UUID/ULID/CUID/numeric identifiers fingerprint identically across users.
*
* Conservative on purpose — only matches segments whose entire content is a
* recognised id format, leaving meaningful path words alone. Used both for
* the page url and for failed-endpoint paths.
*
* Examples:
* /dashboard/vehicles/6a3e7c44-d64b-4245-abc7-adb692a90fff/categories/29ee50e7-887f-4124-b1fc-bbeea5e358d1
* → /dashboard/vehicles/:id/categories/:id
* /api/orders/12345/items → /api/orders/:id/items
* /insights/i/cmpwrb82s002f14fza8lbc7f4 → /insights/i/:id
*/
function normalizePath(u: string | null | undefined): string | null {
if (!u) return u ?? null;
let path: string;
try {
const parsed = new URL(u, "https://x");
path = parsed.pathname || u;
} catch {
const q = u.indexOf("?");
path = q === -1 ? u : u.slice(0, q);
}
return path
.split("/")
.map((seg) => {
if (!seg) return seg;
// UUID v1v8 canonical form
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(seg)) return ":id";
// ULID — Crockford base32, exactly 26 chars
if (/^[0-9A-HJKMNP-TV-Z]{26}$/.test(seg)) return ":id";
// CUID2 / similar — lowercase alphanumeric, length >= 20 starting with a letter
if (/^[a-z][a-z0-9]{19,}$/.test(seg)) return ":id";
// Pure numeric id
if (/^\d+$/.test(seg)) return ":id";
return seg;
})
.join("/");
}
function truncate(s: string, n: number): string { function truncate(s: string, n: number): string {
return s.length > n ? `${s.slice(0, n)}` : s; return s.length > n ? `${s.slice(0, n)}` : s;
} }

View File

@@ -0,0 +1,57 @@
/**
* Smoke test for the insight dedup decision logic (pure helpers, no I/O).
* Run: pnpm --filter worker exec tsx src/lib/dedup.smoke.ts
*
* Covers the two new guarantees:
* 1. No duplicate insights — same-fingerprint / aliased sessions never create
* a new row (attach/reanalyze), and a confident semantic match attaches.
* 2. No insights similar to dismissed ones — dismissed/duplicate fingerprints
* are SUPPRESSED, never re-opened.
*/
import { classifyExistingAction, shouldAcceptMatch, type ExistingAction } from "./dedup";
let pass = 0;
let fail = 0;
function eq<T>(label: string, got: T, want: T): void {
if (got === want) {
pass++;
} else {
fail++;
console.error(`${label}: got ${JSON.stringify(got)} want ${JSON.stringify(want)}`);
}
}
// ─── classifyExistingAction(status, isStale, allowStaleReanalyze) ───
const cea = (s: string, stale: boolean, allow: boolean): ExistingAction =>
classifyExistingAction(s, stale, allow);
// Suppressed themes are never re-opened, regardless of staleness/allow flag.
eq("dismissed → suppress (fresh)", cea("dismissed", false, true), "suppress");
eq("dismissed → suppress (stale)", cea("dismissed", true, true), "suppress");
eq("dismissed → suppress (gate path)", cea("dismissed", true, false), "suppress");
eq("duplicate → suppress", cea("duplicate", true, true), "suppress");
// Fixed themes that recur are flagged as regressions.
eq("validated → regress", cea("validated", false, true), "regress");
eq("shipped → regress (stale)", cea("shipped", true, true), "regress");
// Active themes accumulate; stale ones refresh only on the Layer-1 path.
eq("new fresh → attach", cea("new", false, true), "attach");
eq("new stale (Layer1) → reanalyze", cea("new", true, true), "reanalyze");
eq("new stale (gate, no-reanalyze) → attach", cea("new", true, false), "attach");
eq("in_backlog fresh → attach", cea("in_backlog", false, true), "attach");
eq("in_progress stale (Layer1) → reanalyze", cea("in_progress", true, true), "reanalyze");
eq("regressed fresh → attach", cea("regressed", false, true), "attach");
eq("triaged stale (gate) → attach", cea("triaged", true, false), "attach");
// ─── shouldAcceptMatch(matchId, confidence, catalogIds, threshold) ───
const cat = new Set(["cm_a", "cm_b", "cm_c"]);
const T = 0.72;
eq("null match → reject", shouldAcceptMatch(null, 0.99, cat, T), false);
eq("hallucinated id → reject", shouldAcceptMatch("cm_zzz", 0.99, cat, T), false);
eq("in-catalog + above threshold → accept", shouldAcceptMatch("cm_a", 0.8, cat, T), true);
eq("in-catalog + at threshold → accept", shouldAcceptMatch("cm_b", 0.72, cat, T), true);
eq("in-catalog + below threshold → reject", shouldAcceptMatch("cm_c", 0.71, cat, T), false);
console.log(`\ndedup smoke: ${pass} passed, ${fail} failed`);
if (fail > 0) process.exit(1);

View File

@@ -0,0 +1,160 @@
import { prisma } from "../db";
import { callDeepSeek, extractJson, type CallResult } from "./deepseek";
// ─── Status families (kept in sync with analyze.ts) ───
export const ACTIVE_STATUSES = ["new", "triaged", "in_backlog", "in_progress", "regressed"];
export const SUPPRESSED_STATUSES = ["dismissed", "duplicate"];
export const FIXED_STATUSES = ["validated", "shipped"];
// ─── Tunables ───
export const DEDUP_ENABLED = (process.env.INSIGHT_DEDUP_GATE ?? "true") !== "false";
export const DEDUP_THRESHOLD = Number(process.env.INSIGHT_DEDUP_THRESHOLD ?? "0.72");
const SUPPRESS_LOOKBACK_DAYS = Number(process.env.INSIGHT_DEDUP_SUPPRESS_DAYS ?? "90");
const CATALOG_MAX = Number(process.env.INSIGHT_DEDUP_CATALOG_MAX ?? "120");
export type CatalogEntry = { id: string; title: string; type: string; status: string };
// ─── Pure decision helpers (unit-testable, no I/O) ───
export type ExistingAction = "suppress" | "regress" | "attach" | "reanalyze";
/**
* Decide what to do when a session's fingerprint already maps to an insight.
* suppress — dismissed/duplicate: bump counters, keep status (no LLM)
* regress — validated/shipped: flag regression (no LLM)
* reanalyze — active but stale: caller re-runs the LLM to refresh
* attach — active and fresh: bump counters (no LLM)
*/
export function classifyExistingAction(
status: string,
isStale: boolean,
allowStaleReanalyze: boolean,
): ExistingAction {
if (SUPPRESSED_STATUSES.includes(status)) return "suppress";
if (FIXED_STATUSES.includes(status)) return "regress";
if (allowStaleReanalyze && isStale) return "reanalyze";
return "attach";
}
/**
* Decide whether to accept a dedup-gate match: the id must exist in the catalog
* (anti-hallucination) and confidence must clear the threshold.
*/
export function shouldAcceptMatch(
matchId: string | null,
confidence: number,
catalogIds: Set<string>,
threshold: number,
): boolean {
return !!matchId && catalogIds.has(matchId) && confidence >= threshold;
}
/**
* Build the dedup catalog the LLM gate compares a new session against:
* - ALL currently-active insights (dedup target → attach).
* - Recently-touched suppressed/fixed insights (dismissed / duplicate /
* validated / shipped) within the lookback window (suppression target →
* don't recreate a known non-issue / flag regression).
* Active entries are prioritised, then most-recent others, capped at CATALOG_MAX
* so the prompt stays bounded as the dismissed pile grows over time.
*/
export async function buildInsightCatalog(projectKey: string): Promise<CatalogEntry[]> {
const since = new Date(Date.now() - SUPPRESS_LOOKBACK_DAYS * 86_400_000);
const rows = await prisma.insight.findMany({
where: {
projectKey,
OR: [
{ status: { in: ACTIVE_STATUSES } },
{ status: { in: [...SUPPRESSED_STATUSES, ...FIXED_STATUSES] }, updatedAt: { gte: since } },
],
},
select: { id: true, title: true, type: true, status: true },
orderBy: [{ updatedAt: "desc" }],
});
const active = rows.filter((r) => ACTIVE_STATUSES.includes(r.status));
const rest = rows.filter((r) => !ACTIVE_STATUSES.includes(r.status));
return [...active, ...rest].slice(0, CATALOG_MAX);
}
const SYSTEM_PROMPT = `You are a strict deduplication classifier for a product-analytics insight pipeline (Sase.tr — a Turkish car-parts catalog SaaS where users decode a VIN, browse a category tree / parts panel, and subscribe via Stripe).
You receive ONE new user-session signal and a CATALOG of existing insights, each tagged with a status. Decide whether the new session describes the SAME underlying product problem as exactly one catalog entry.
"Same problem" means: same root cause AND same user-facing failure on the same surface/flow — e.g. both are "category-tree navigation produces rage-clicks", or both are "VIN decode upstream provider timeout".
NOT the same: merely sharing a page, a severity, or the generic signal "ux_friction". Distinct sub-problems on the same page are DIFFERENT (e.g. a broken schema IMAGE vs. a slow category LIST vs. a parts panel that won't SELECT are three different insights).
Matching an entry whose status is "dismissed" or "duplicate" is expected and valuable — it means this is a known non-issue we must NOT recreate. Matching "validated"/"shipped" means a previously-fixed problem may have regressed.
Return ONLY JSON: {"match_insight_id": "<id from catalog, or null>", "confidence": <0..1>, "reason": "<one short sentence>"}.
Be conservative: when in doubt, return null with low confidence. Never invent an id that is not in the catalog.`;
export type DedupResult = {
matchId: string | null;
confidence: number;
reason: string;
raw: CallResult | null;
};
/**
* Ask the flash model whether this session matches an existing insight.
* Returns matchId=null if the catalog is empty, parsing fails, or the model
* returns an id that is not actually in the catalog (anti-hallucination guard).
*/
export async function findSemanticMatch(input: {
timeline: string;
tags: string[];
catalog: CatalogEntry[];
}): Promise<DedupResult> {
if (input.catalog.length === 0) {
return { matchId: null, confidence: 0, reason: "empty catalog", raw: null };
}
const catalogText = input.catalog
.map((c) => `- [${c.id}] (status=${c.status}, type=${c.type}) ${c.title}`)
.join("\n");
const snippet =
input.timeline.length > 2800 ? `${input.timeline.slice(0, 2800)}\n…(truncated)` : input.timeline;
const userPrompt = [
"=== NEW SESSION ===",
`tags: [${input.tags.join(", ")}]`,
"",
snippet,
"",
"=== EXISTING INSIGHTS CATALOG ===",
catalogText,
"",
'Return JSON only: {"match_insight_id": <id|null>, "confidence": 0..1, "reason": "..."}',
].join("\n");
let raw: CallResult;
try {
raw = await callDeepSeek({
tier: "flash",
systemPrompt: SYSTEM_PROMPT,
userPrompt,
maxOutputTokens: 200,
temperature: 0,
});
} catch (e) {
return { matchId: null, confidence: 0, reason: `dedup call failed: ${(e as Error).message}`, raw: null };
}
try {
const p = JSON.parse(extractJson(raw.text)) as {
match_insight_id?: string | null;
confidence?: number;
reason?: string;
};
let matchId = p.match_insight_id ?? null;
let confidence = typeof p.confidence === "number" ? p.confidence : 0;
let reason = String(p.reason ?? "");
// Anti-hallucination: only accept ids that are actually in the catalog.
if (matchId && !input.catalog.some((c) => c.id === matchId)) {
reason = `rejected non-catalog id ${matchId}; ${reason}`;
matchId = null;
confidence = 0;
}
return { matchId, confidence, reason, raw };
} catch (e) {
return { matchId: null, confidence: 0, reason: `parse error: ${(e as Error).message}`, raw };
}
}

View File

@@ -47,6 +47,23 @@ export const TRACKED_EVENTS: string[] = [
"parts_export_completed", "parts_export_completed",
"oem_code_copied", "oem_code_copied",
// Catalog / category browsing — Sase.tr's core value path (vehicle → categories
// → parts). These were previously NOT fetched, so a vehicle that resolved but
// rendered no parts/categories (the #1 churn complaint) was invisible to the
// tagger. `empty_catalog_cta_clicked` / `parts_panel_viewed.parts_count=0` are
// the concrete empty-state signals; the rest let us tell "drilled in but got
// nothing" apart from "decoded and bounced".
"catalog_search_opened",
"catalog_brands_viewed",
"catalog_brand_clicked",
"catalog_subcatalog_selected",
"catalog_models_viewed",
"catalog_model_clicked",
"catalog_locked_brand_upgrade_clicked",
"category_view_changed",
"empty_catalog_cta_clicked",
"part_reference_clicked",
// Payment (v1 + v2) // Payment (v1 + v2)
"payment_initiated", "payment_initiated",
"payment_success", "payment_success",

View File

@@ -98,6 +98,7 @@ export function customEventPromoteReasons(eventNames: string[]): string[] {
"subscription_cancelled", "subscription_cancelled",
"trial_urgency_banner_cta_clicked", "trial_urgency_banner_cta_clicked",
"downgrade_offer_shown", "downgrade_offer_shown",
"empty_catalog_cta_clicked", // resolved a vehicle but the catalog was empty
]; ];
for (const e of single) if (set.has(e)) reasons.push(`event:${e}`); for (const e of single) if (set.has(e)) reasons.push(`event:${e}`);

View File

@@ -264,17 +264,25 @@ Return JSON per the schema. Use occurrence patterns from the bundle summary to e
}, },
{ {
tag: "provider_quality", tag: "provider_quality",
version: 3, version: 4,
name: "Provider Quality v3 (TR, ext maxLen)", name: "Provider Quality v4 (TR, client-validation guardrail)",
systemPrompt: `You analyze upstream provider failures (PL24/PCAT/RMEX/TecDoc) impacting Sase.tr users. Identify which provider failed and propose action. systemPrompt: `You analyze upstream provider failures (PL24/PCAT/RMEX/TecDoc) impacting Sase.tr users. Identify which provider failed and propose action.
${SASE_CONTEXT} ${SASE_CONTEXT}
CRITICAL GUARDRAIL — distinguish client-side input rejection from upstream provider failure:
- A \`vin_decode_failed\` event is ONLY a provider issue when the input actually reached a provider. Check the inlined event properties:
- If \`error\` contains "Geçersiz şase numarası", "17 karakter olmalı", "I, O, Q", "invalid VIN", or any 17-character / format complaint → this was a **client-side validation reject**, the server was NEVER called.
- If \`vin\` is shorter than 17 characters, contains spaces (e.g. "5Q0 907 521" — that's a VW PART NUMBER, not a VIN), or contains I/O/Q → same: client-side rejection.
- If \`source\` is a UI location ("landing", "dashboard") and no \`provider_attempted\` field is present → did not reach a provider.
- In all of the above cases this is **NOT a provider_quality issue**. Return \`confidence: 0.15\`, \`failure_mode: "unknown"\`, \`affected_provider: "multi"\`, and in the \`hypothesis\` explicitly state: "Bu provider hatası değil — kullanıcı VIN alanına geçersiz format girdi (client-side reddi). Doğru kategori ux_friction / input affordance." This low-confidence output is preferable to inventing a provider issue.
- Only return a high-confidence provider_quality verdict when at least 2 events show \`provider_attempted\` set OR the timeline shows network 5xx/timeout patterns from upstream endpoints (e.g. \`/api/vin/decode\`, \`/api/parts\`, requests to PL24/PCAT/RMEX/TecDoc paths).
Schema: Schema:
${JSON.stringify(PROVIDER_SCHEMA)}`, ${JSON.stringify(PROVIDER_SCHEMA)}`,
userPromptTemplate: `{{timeline}} userPromptTemplate: `{{timeline}}
Return JSON per the schema.`, Return JSON per the schema. Before classifying as provider issue, verify the guardrail above by checking inlined event properties (error, source, vin).`,
outputSchemaJson: PROVIDER_SCHEMA, outputSchemaJson: PROVIDER_SCHEMA,
modelTier: "flash", modelTier: "flash",
maxOutputTokens: 800, maxOutputTokens: 800,
@@ -299,6 +307,12 @@ export function pickPromptTag(tags: string[]): string {
// Upgrade hesitation — pricing page concerns // Upgrade hesitation — pricing page concerns
if (set.has("upgrade_hesitation")) return "upgrade_hesitation"; if (set.has("upgrade_hesitation")) return "upgrade_hesitation";
// Client-side VIN validation rejects must route to ux_friction — these are
// input affordance problems (user typed a part number / short string into the
// VIN field), not upstream provider failures. Checked *before* provider tags
// so that the more specific signal wins.
if (set.has("vin_decode_client_validation_fail")) return "ux_friction";
// Provider issues // Provider issues
if ( if (
set.has("provider_reliability_issue") || set.has("provider_reliability_issue") ||
@@ -308,8 +322,16 @@ export function pickPromptTag(tags: string[]): string {
) )
return "provider_quality"; return "provider_quality";
// Bugs // Bugs (incl. core-value parts/category render failures — bug_triage carries
if (set.has("bug_suspected") || set.has("server_error_impact")) return "bug_triage"; // is_likely_provider_issue / implicated_provider so the model can attribute an
// empty catalog to a provider data gap vs. a render/query defect).
if (
set.has("bug_suspected") ||
set.has("server_error_impact") ||
set.has("parts_render_blocked") ||
set.has("catalog_empty_result")
)
return "bug_triage";
// Onboarding // Onboarding
if (set.has("onboarding_stuck")) return "onboarding_stuck"; if (set.has("onboarding_stuck")) return "onboarding_stuck";

View File

@@ -0,0 +1,198 @@
/**
* Smoke test for the tagger search-affordance-misuse suppression.
* Run: pnpm --filter worker exec tsx src/lib/tagger.smoke.ts
*
* Guarantees the new guard is SURGICAL: it silences generic ux_friction /
* frustrated_session ONLY for "user fiddling with the VIN search box, nothing
* broken" sessions, while every concrete failure stays actionable.
*/
import type { SessionMeta } from "@prisma/client";
import type { CanonicalEvent } from "./event-taxonomy";
import { tagSession } from "./tagger";
let pass = 0;
let fail = 0;
function assert(label: string, cond: boolean): void {
if (cond) pass++;
else {
fail++;
console.error(`${label}`);
}
}
const ev = (name: string, properties: Record<string, unknown> = {}): CanonicalEvent =>
({ name, properties, rawName: name } as unknown as CanonicalEvent);
function mkSession(over: Partial<SessionMeta>): SessionMeta {
return {
errorCount: 0,
rageClickCount: 0,
deadClickCount: 0,
network5xxCount: 0,
startUrl: "https://sase.tr/dashboard/search",
durationMs: 200_000,
isAuthenticated: true,
clickCount: 5,
subscriptionTier: null,
startedAt: new Date(1_700_000_000_000),
...over,
} as unknown as SessionMeta;
}
const ctx = (events: CanonicalEvent[], userProperties: Record<string, unknown> = {}) =>
({ customEvents: events, userProperties, groupProperties: null });
const tags = (s: SessionMeta, events: CanonicalEvent[], up: Record<string, unknown> = {}) =>
tagSession(s, ctx(events, up)).tags;
// 1) The reported case: VIN search-box misuse, 17 rage clicks, NOTHING broken → suppressed.
{
const t = tags(
mkSession({ rageClickCount: 17, errorCount: 0, network5xxCount: 0 }),
[ev("search_input_focused"), ev("search_history_item_selected"), ev("parts_panel_viewed")],
);
assert("misuse: no ux_friction", !t.includes("ux_friction"));
assert("misuse: no frustrated_session", !t.includes("frustrated_session"));
assert("misuse: tag-less → will be discarded", t.length === 0);
}
// 2) Real JS errors + rage → still bug_suspected (misuse guard off when errorCount>0).
{
const t = tags(
mkSession({ rageClickCount: 5, errorCount: 2 }),
[ev("search_input_focused")],
);
assert("js-error: bug_suspected kept", t.includes("bug_suspected"));
}
// 3) Server 5xx storm → server_error_impact kept (misuse guard off when 5xx>0).
{
const t = tags(
mkSession({ rageClickCount: 4, network5xxCount: 3 }),
[ev("search_input_focused")],
);
assert("5xx: server_error_impact kept", t.includes("server_error_impact"));
}
// 4) Payment friction on a search session → conversion signal kept (misuse guard off).
{
const t = tags(
mkSession({ rageClickCount: 4 }),
[ev("search_input_focused"), ev("payment_initiated")],
);
assert("payment: payment_friction kept", t.includes("payment_friction"));
assert("payment: frustrated_session NOT suppressed", t.includes("frustrated_session"));
}
// 5) Category-tree rage WITHOUT the search box → still surfaces (scope = search box only).
{
const t = tags(
mkSession({ rageClickCount: 10, startUrl: "https://sase.tr/dashboard/vehicles/x/categories/y" }),
[ev("parts_panel_viewed")],
);
assert("category: ux_friction kept (no search_input_focused)", t.includes("ux_friction"));
assert("category: frustrated_session kept", t.includes("frustrated_session"));
}
// 6) Real upstream VIN decode failures on a search session → provider signal kept.
{
const t = tags(
mkSession({ rageClickCount: 3 }),
[
ev("search_input_focused"),
ev("vin_decode_failed", { provider_attempted: "PL24" }),
ev("vin_decode_failed", { provider_attempted: "PL24" }),
],
);
assert("vin-fail: vin_decode_fail_pattern kept", t.includes("vin_decode_fail_pattern"));
}
// 7) Search box used but ≥3 validation failures → that has its own P3 signal, not suppressed.
{
const t = tags(
mkSession({ rageClickCount: 6 }),
[
ev("search_input_focused"),
ev("search_input_validation_failed"),
ev("search_input_validation_failed"),
ev("search_input_validation_failed"),
],
);
assert("validation-friction: search_validation_friction kept", t.includes("search_validation_friction"));
assert("validation-friction: frustrated_session NOT suppressed", t.includes("frustrated_session"));
}
// 8) THE serkan case: signed-up trial user, VIN decodes (sees model), drills the
// category tree + clicks a catalog model, but parts NEVER render — zero rage,
// zero error, zero 5xx. A demo category that *does* return parts must not mask
// the real-page failure, and touching the search box must not silence it.
{
const t = tags(
mkSession({ rageClickCount: 0, errorCount: 0, network5xxCount: 0 }),
[
ev("search_input_focused"),
ev("vin_decode_succeeded"),
ev("category_view_changed"),
ev("category_view_changed"),
ev("category_view_changed"),
ev("catalog_model_clicked", { brand_name: "Ford" }),
// demo categories that DO return parts — must be excluded from the signal:
ev("parts_panel_viewed", { parts_count: 13, $current_url: "https://sase.tr/demo/categories/x" }),
],
);
assert("serkan: parts_render_blocked fired", t.includes("parts_render_blocked"));
assert("serkan: was caught (not tag-less / not search-misuse suppressed)", t.length > 0);
}
// 9) Parts panel renders EMPTY on a real dashboard page (parts_count=0, never non-empty).
{
const t = tags(mkSession({}), [
ev("vin_decode_succeeded"),
ev("parts_panel_viewed", {
parts_count: 0,
$current_url: "https://sase.tr/dashboard/vehicles/abc/categories/def",
}),
]);
assert("empty-panel: catalog_empty_result fired", t.includes("catalog_empty_result"));
assert("empty-panel: not falsely 'blocked' (panel did render)", !t.includes("parts_render_blocked"));
}
// 10) Explicit empty-state CTA → concrete empty result.
{
const t = tags(mkSession({}), [
ev("vin_decode_succeeded"),
ev("empty_catalog_cta_clicked", { vehicle_label: "Ford Focus", category_name: "mekanik" }),
]);
assert("empty-cta: catalog_empty_result fired", t.includes("catalog_empty_result"));
}
// 11) Happy path: parts actually render (count>0) → no parts-failure tags.
{
const url = "https://sase.tr/dashboard/vehicles/abc/categories/def";
const t = tags(mkSession({}), [
ev("vin_decode_succeeded"),
ev("catalog_model_clicked"),
ev("parts_panel_viewed", { parts_count: 20, $current_url: url }),
ev("oem_code_copied", { $current_url: url }),
]);
assert("happy: no parts_render_blocked", !t.includes("parts_render_blocked"));
assert("happy: no catalog_empty_result", !t.includes("catalog_empty_result"));
}
// 12) Decode-and-bounce (no drilling toward parts) → no false positive.
{
const t = tags(mkSession({}), [ev("vin_decode_succeeded")]);
assert("bounce: no parts_render_blocked", !t.includes("parts_render_blocked"));
}
// 13) Locked brand → upgrade prompt: paywall, not a defect.
{
const t = tags(mkSession({}), [
ev("vin_decode_succeeded"),
ev("catalog_model_clicked", { brand_name: "Mercedes" }),
ev("catalog_locked_brand_upgrade_clicked", { brand_name: "Mercedes" }),
]);
assert("locked: no parts_render_blocked (paywall, not bug)", !t.includes("parts_render_blocked"));
}
console.log(`\ntagger smoke: ${pass} passed, ${fail} failed`);
if (fail > 0) process.exit(1);

View File

@@ -47,6 +47,80 @@ export function tagSession(s: SessionMeta, ctx: EnrichmentCtx | null): TagResult
const userProps = ctx?.userProperties ?? {}; const userProps = ctx?.userProperties ?? {};
const groupProps = ctx?.groupProperties ?? null; const groupProps = ctx?.groupProperties ?? null;
// ─── Parts / category render signals (Sase.tr core-value path) ───
// The product's whole job: resolve a vehicle (VIN or catalog) → list its
// categories + parts. When that final step yields *nothing* the product has
// failed for the user even though VIN decode "succeeded" and no JS error/5xx
// fired — the catalog returned an empty panel, 0 models, or the user hit the
// explicit empty-state CTA. This used to be invisible (the catalog_* family
// wasn't even fetched), so a real prospect could browse Ford/Opel, see no
// parts, and churn with no insight raised. Scoped to the real /dashboard
// product: /demo is a separate curated marketing surface whose (sometimes
// working) categories must NOT mask a genuine in-product failure.
const inDemo = (e: CanonicalEvent): boolean =>
/\/demo(\/|\?|$)/.test(
String(e.properties["$current_url"] ?? e.properties["$pathname"] ?? ""),
);
const realPartsViews = events.filter((e) => e.name === "parts_panel_viewed" && !inDemo(e));
const successfulPartsView = realPartsViews.some((e) => Number(e.properties.parts_count) > 0);
const emptyPartsView = realPartsViews.some((e) => Number(e.properties.parts_count) === 0);
const emptyModelsList = events.some(
(e) => e.name === "catalog_models_viewed" && !inDemo(e) && Number(e.properties.count) === 0,
);
const explicitEmptyCatalog = events.some(
(e) => e.name === "empty_catalog_cta_clicked" && !inDemo(e),
);
const realOemCopied = events.some((e) => e.name === "oem_code_copied" && !inDemo(e));
// Shape 1 — "the system said: nothing here": panel rendered empty (and never
// non-empty in this session), 0 models listed, or the empty-state CTA shown.
const catalogEmptyResult =
explicitEmptyCatalog || emptyModelsList || (emptyPartsView && !successfulPartsView);
// Shape 2 — "drilled in and got nothing at all": resolved a vehicle and
// actively browsed categories/models but never saw a single part. Gated on
// real engagement (not a decode-and-bounce) and excludes the paywall case
// (locked brand → upgrade prompt, which is a conversion signal, not a defect).
const resolvedVehicle =
has(events, "vin_decode_succeeded") ||
has(events, "vin_decode_candidate_selected") ||
has(events, "catalog_model_clicked");
const browsedForParts =
count(events, "category_view_changed") >= 2 ||
has(events, "catalog_model_clicked") ||
has(events, "catalog_subcatalog_selected");
const partsRenderBlocked =
resolvedVehicle &&
browsedForParts &&
!successfulPartsView &&
!realOemCopied &&
!has(events, "catalog_locked_brand_upgrade_clicked");
// Any genuine parts/category failure disqualifies the search-misuse guard
// below — such a session is a real product defect, never "fiddling with the
// search box, nothing broken".
const partsFailure = catalogEmptyResult || partsRenderBlocked;
// ─── Expected search-box misuse → not insight-worthy ───
// The /dashboard/search box is VIN-only, but users routinely use it to look
// for a part by *name* (e.g. "Cam düğme") or fiddle with search/history and
// rage-click out of affordance confusion — while *nothing is actually broken*
// (no JS errors, no 5xx, no decode/validation/provider failure, no payment).
// These pure-affordance rage sessions are noise, not product defects, so we
// do NOT let the generic ux_friction / frustrated_session tags fire for them;
// with no other actionable tag the session ends up tag-less → discarded
// (no compress / analyze / insight). Real failures still carry a concrete
// event below and stay actionable. Scoped narrowly to search-box sessions on
// purpose, to avoid hiding genuine parts/category bugs.
const searchAffordanceMisuse =
s.errorCount === 0 &&
s.network5xxCount === 0 &&
!partsFailure &&
has(events, "search_input_focused") &&
!has(events, "vin_decode_failed") &&
count(events, "search_input_validation_failed") < 3 &&
!has(events, "payment_initiated") &&
!has(events, "payment_failed") &&
!has(events, "checkout_started");
// ─── Bug detection (rrweb-based, generic fallback) ─── // ─── Bug detection (rrweb-based, generic fallback) ───
if (s.errorCount > 0 && (s.rageClickCount > 0 || s.network5xxCount > 0)) { if (s.errorCount > 0 && (s.rageClickCount > 0 || s.network5xxCount > 0)) {
tags.push("bug_suspected"); tags.push("bug_suspected");
@@ -57,7 +131,7 @@ export function tagSession(s: SessionMeta, ctx: EnrichmentCtx | null): TagResult
severity = bump(severity, "P1"); severity = bump(severity, "P1");
} }
// ─── UX friction ─── // ─── UX friction ───
if (s.errorCount === 0 && (s.rageClickCount > 0 || s.deadClickCount > 0)) { if (s.errorCount === 0 && (s.rageClickCount > 0 || s.deadClickCount > 0) && !searchAffordanceMisuse) {
tags.push("ux_friction"); tags.push("ux_friction");
severity = bump(severity, "P2"); severity = bump(severity, "P2");
} }
@@ -65,29 +139,65 @@ export function tagSession(s: SessionMeta, ctx: EnrichmentCtx | null): TagResult
// ─── Frustrated session ─── // ─── Frustrated session ───
// Sustained rage clicking (3+ clusters) is a stronger signal than a single // Sustained rage clicking (3+ clusters) is a stronger signal than a single
// cluster — promote it past ux_friction so it surfaces above generic noise. // cluster — promote it past ux_friction so it surfaces above generic noise.
if (s.rageClickCount >= 3) { if (s.rageClickCount >= 3 && !searchAffordanceMisuse) {
tags.push("frustrated_session"); tags.push("frustrated_session");
severity = bump(severity, "P2"); severity = bump(severity, "P2");
} }
// ─── VIN decode failure pattern ─── // ─── VIN decode failure pattern ───
// Distinguish *client-side validation rejects* (input too short / wrong format
// / forbidden chars I,O,Q — server never called) from *upstream provider
// failures* (PL24/PCAT/RMEX/TecDoc timeout/error). Bundling them together
// produces false-positive "provider issue" insights (e.g. user typing a VW
// part number "5Q0 907 521" into the VIN field hits client-side regex; no
// provider was contacted, so it isn't a provider quality signal).
const vinFails = events.filter((e) => e.name === "vin_decode_failed"); const vinFails = events.filter((e) => e.name === "vin_decode_failed");
if (vinFails.length >= 2) { if (vinFails.length >= 1) {
const providers = new Set( const isClientValidationReject = (e: CanonicalEvent): boolean => {
vinFails.map((e) => String(e.properties.provider_attempted ?? e.properties.source ?? "")), const p = e.properties ?? {};
); const err = String(p.error ?? "");
if (providers.size === 1 && [...providers][0]) { const vin = String(p.vin ?? "");
tags.push("vin_decode_fail_pattern"); // Frontend Zod/regex messages we ship — keep in sync with web VIN validator.
severity = bump(severity, "P1"); const clientMsgRe = /(Geçersiz şase|17 karakter|I, O, Q|invalid VIN|must be 17|format)/i;
} else { if (clientMsgRe.test(err)) return true;
// Different providers failing → still notable // Length / forbidden-char heuristic — if the user typed something that
tags.push("vin_decode_repeated_failure"); // couldn't possibly reach the upstream, treat as client-side rejection.
severity = bump(severity, "P2"); if (vin && vin.replace(/\s/g, "").length !== 17) return true;
if (vin && /[IOQ]/i.test(vin)) return true;
return false;
};
const clientRejects = vinFails.filter(isClientValidationReject);
const realFailures = vinFails.filter((e) => !isClientValidationReject(e));
if (clientRejects.length >= 1 && realFailures.length === 0) {
// Pure client-side input affordance problem — route through ux_friction,
// not provider_quality. Severity is low (no service impact).
tags.push("vin_decode_client_validation_fail");
severity = bump(severity, "P3");
} else if (realFailures.length >= 2) {
// True upstream failures: group by *provider* (not the UI source field —
// "landing"/"dashboard" are page locations, not providers).
const providers = new Set(
realFailures
.map((e) => String(e.properties.provider_attempted ?? ""))
.filter((v) => v.length > 0),
);
if (providers.size === 1) {
tags.push("vin_decode_fail_pattern");
severity = bump(severity, "P1");
} else if (providers.size > 1) {
tags.push("vin_decode_repeated_failure");
severity = bump(severity, "P2");
} else {
// Unknown provider attribution but server-side failure shape — still
// worth surfacing but as a softer signal.
tags.push("vin_decode_repeated_failure");
severity = bump(severity, "P2");
}
} else if (realFailures.length === 1) {
tags.push("vin_decode_failed_single");
severity = bump(severity, "P3");
} }
} else if (vinFails.length === 1) {
// Single failure is still a quality signal, less severe
tags.push("vin_decode_failed_single");
severity = bump(severity, "P3");
} }
if (has(events, "provider_fallback_triggered")) { if (has(events, "provider_fallback_triggered")) {
@@ -143,6 +253,19 @@ export function tagSession(s: SessionMeta, ctx: EnrichmentCtx | null): TagResult
severity = bump(severity, "P3"); severity = bump(severity, "P3");
} }
// ─── Parts / category render failure (signals computed at top) ───
// Core-value failure: the user got a vehicle but no parts/categories. This is
// the highest-intent churn signal Sase.tr has — surface it as a concrete bug,
// not generic friction.
if (catalogEmptyResult) {
tags.push("catalog_empty_result");
severity = bump(severity, "P2");
}
if (partsRenderBlocked) {
tags.push("parts_render_blocked");
severity = bump(severity, "P1");
}
// ─── Search friction ─── // ─── Search friction ───
if (count(events, "search_input_validation_failed") >= 3) { if (count(events, "search_input_validation_failed") >= 3) {
tags.push("search_validation_friction"); tags.push("search_validation_friction");

View File

@@ -72,6 +72,33 @@ export function alertP0Insight(opts: {
return sendTelegram({ text, dedupeKey: `p0:${opts.insightId}` }); return sendTelegram({ text, dedupeKey: `p0:${opts.insightId}` });
} }
export function alertCatalogGap(opts: {
brand: string;
failures: number;
uniqueUsers: number;
windowDays: number;
severity: string;
insightId: string;
panelUrl: string;
day: string;
regressed?: boolean;
}): Promise<TelegramSendResult> {
const head = opts.regressed
? `↩️ <b>Katalog açığı GERİ DÖNDÜ</b> [${opts.severity}]`
: `🗂️ <b>Katalog kapsama açığı</b> [${opts.severity}]`;
const text = [
head,
`<b>${escapeHtml(opts.brand)}</b>: ${opts.failures} başarısız sorgu / ${opts.uniqueUsers} kullanıcı (son ${opts.windowDays}g)`,
`Araç tanınıyor ama parça kataloğu yok → kullanıcı parça göremiyor.`,
``,
`<a href="${opts.panelUrl}/insights/i/${opts.insightId}">Insight</a> · <a href="${opts.panelUrl}/projects/sase/vin-decode">VIN dashboard</a>`,
].join("\n");
return sendTelegram({
text,
dedupeKey: `catalog_gap:${opts.regressed ? "regress" : "new"}:${opts.brand}:${opts.day}`,
});
}
export function alertRegression(opts: { export function alertRegression(opts: {
insightId: string; insightId: string;
title: string; title: string;

View File

@@ -49,11 +49,12 @@ export async function startContentPipeline() {
{ name: "content-generate", data: {}, opts: { removeOnComplete: 50, removeOnFail: 25 } }, { name: "content-generate", data: {}, opts: { removeOnComplete: 50, removeOnFail: 25 } },
); );
new Worker(QUEUE, runJob, { const worker = new Worker(QUEUE, runJob, {
connection: redis, connection: redis,
concurrency: 1, concurrency: 1,
lockDuration: 5 * 60_000, lockDuration: 5 * 60_000,
stalledInterval: 60_000, stalledInterval: 60_000,
}); });
console.log("[content] armed: content-topics@*/8h, content-generate@*/10min"); console.log("[content] armed: content-topics@*/8h, content-generate@*/10min");
return worker;
} }

View File

@@ -46,6 +46,15 @@ export async function startScheduledJobs() {
{ name: "panel-backup", data: {}, opts: { removeOnComplete: 30, removeOnFail: 30 } }, { name: "panel-backup", data: {}, opts: { removeOnComplete: 30, removeOnFail: 30 } },
); );
new Worker(QUEUE, runJob, { connection: redis, concurrency: 1 }); const worker = new Worker(QUEUE, runJob, {
connection: redis,
concurrency: 1,
// panel-backup (pg_dump) can run longer than the 30s default lock; a too-short
// lock makes the job look "stalled", gets re-run, and the original then fails
// its moveToFinished with "Missing lock". Match the other queues' 5min lock.
lockDuration: 5 * 60_000,
stalledInterval: 60_000,
});
console.log("[scheduler] armed: nightly-refresh@03:00, audit-archive@03:30, panel-backup@04:00"); console.log("[scheduler] armed: nightly-refresh@03:00, audit-archive@03:30, panel-backup@04:00");
return worker;
} }

View File

@@ -10,6 +10,7 @@ import { runRetention } from "../jobs/retention";
import { runEvalSet } from "../jobs/eval-run"; import { runEvalSet } from "../jobs/eval-run";
import { runDailyBrief } from "../jobs/daily-brief"; import { runDailyBrief } from "../jobs/daily-brief";
import { runVinAnomalyDetect } from "../jobs/vin-anomaly"; import { runVinAnomalyDetect } from "../jobs/vin-anomaly";
import { runCatalogGapDetect } from "../jobs/catalog-gap-detect";
import { runArchivePosthogEvents } from "../jobs/posthog-event-archive"; import { runArchivePosthogEvents } from "../jobs/posthog-event-archive";
import { runArchiveRecordings } from "../jobs/archive-recordings"; import { runArchiveRecordings } from "../jobs/archive-recordings";
import { runArchiveIdentity } from "../jobs/posthog-identity-archive"; import { runArchiveIdentity } from "../jobs/posthog-identity-archive";
@@ -91,6 +92,15 @@ async function runJob(job: Job) {
} }
return res; return res;
} }
case "catalog-gap-detect": {
const res = await runCatalogGapDetect();
if (res.created > 0 || res.updated > 0 || !res.ok) {
console.log(
`[pipeline] catalog-gap gaps=${res.gaps} created=${res.created} updated=${res.updated} alerts=${res.alertsFired}${res.reason ? ` reason=${res.reason}` : ""}`,
);
}
return res;
}
case "posthog-event-archive": { case "posthog-event-archive": {
const res = await runArchivePosthogEvents(); const res = await runArchivePosthogEvents();
if (res.fetched > 0 || res.dumped > 0 || res.error) { if (res.fetched > 0 || res.dumped > 0 || res.error) {
@@ -181,6 +191,11 @@ export async function startInsightPipeline() {
{ pattern: "*/5 * * * *" }, { pattern: "*/5 * * * *" },
{ name: "vin-anomaly-detect", data: {}, opts: { removeOnComplete: 30, removeOnFail: 15 } }, { name: "vin-anomaly-detect", data: {}, opts: { removeOnComplete: 30, removeOnFail: 15 } },
); );
await queue.upsertJobScheduler(
"catalog-gap-detect",
{ pattern: "30 */6 * * *" },
{ name: "catalog-gap-detect", data: {}, opts: { removeOnComplete: 30, removeOnFail: 15 } },
);
await queue.upsertJobScheduler( await queue.upsertJobScheduler(
"posthog-event-archive", "posthog-event-archive",
{ pattern: "*/15 * * * *" }, { pattern: "*/15 * * * *" },
@@ -202,13 +217,14 @@ export async function startInsightPipeline() {
{ name: "sentry-archive", data: {}, opts: { removeOnComplete: 30, removeOnFail: 15 } }, { name: "sentry-archive", data: {}, opts: { removeOnComplete: 30, removeOnFail: 15 } },
); );
new Worker(QUEUE, runJob, { const worker = new Worker(QUEUE, runJob, {
connection: redis, connection: redis,
concurrency: 2, concurrency: 2,
lockDuration: 5 * 60_000, lockDuration: 5 * 60_000,
stalledInterval: 60_000, stalledInterval: 60_000,
}); });
console.log( console.log(
"[pipeline] armed: posthog-ingest@*/5min, tag-sessions@*/2min, compress-sessions@*/3min, analyze@*/4min, validation@05:00, github-sync@*/10min, retention@04:15, daily-brief@05:00, vin-anomaly-detect@*/5min, posthog-event-archive@*/15min, archive-recordings@*/6h, posthog-identity-archive@03:00, sentry-archive@hourly", "[pipeline] armed: posthog-ingest@*/5min, tag-sessions@*/2min, compress-sessions@*/3min, analyze@*/4min, validation@05:00, github-sync@*/10min, retention@04:15, daily-brief@05:00, vin-anomaly-detect@*/5min, catalog-gap-detect@*/6h, posthog-event-archive@*/15min, archive-recordings@*/6h, posthog-identity-archive@03:00, sentry-archive@hourly",
); );
return worker;
} }