root/sp
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

fix(insights): tighten session fingerprint so duplicate UX issues dedupe #5

Merged
root merged 3 commits from fix/insight-fingerprint-dedupe into main 2026-06-03 20:47:12 +03:00
Conversation 0 Commits 3 Files Changed 8 +538 -57
root commented 2026-06-02 19:44:10 +03:00
Owner
Copy Link

ℹ️ Stack notu

Bu PR fix/insight-vin-validation-distinction (PR #4) branch'i üzerine kuruldu çünkü PR #4 henüz merge edilmemişti. Sonuç: PR #5 hem PR #4'ün düzeltmesini hem de fingerprint dedupe düzeltmesini içeriyor. İki ayrı commit olarak görünür:

  1. 9a479f9 — fix(insights): distinguish client-side VIN validation rejects from provider failures (PR #4 ile aynı)
  2. a7fe80f — fix(insights): tighten session fingerprint so duplicate UX issues dedupe (bu PR'ın asıl içeriği)

Yani PR #5 merge edilirse PR #4'e gerek kalmaz — PR #4 manuel olarak kapatılabilir.

Sorun

Aynı kök sebepli UX problemleri panel'de tek bir Insight olarak dedupe edilmiyor — son 1 günde 6 farklı new insight açılmış, hepsi "kategori ağacında bouncing → parts panel'i bulamama → rage click" senaryosunu farklı kelimelerle anlatıyor:

id severity başlık occ
cmpclg7b9000dflajkg27hxvt P2 Kategori ağacında rage click'ler → sase.tr#76 97
cmp5yj1pa00071bakh9nqdh22 P1 VIN sonrası parça paneli sürekli yeniden yükleniyor 34
cmpwrb82s002f14fza8lbc7f4 P1 Parça arama filtresinde VIN girişi sonrası tıkanıklık 1
cmpwqgbyc002d14fzhnumfatw P2 Kategori sayfasında tıklanabilir öğe bulunamıyor 1
cmpwlwt4a002314fzg8wyi3nh P2 Kategori ağacında rage click + boş sayfa 1
cmpoeec4l001j14ozjjov5vrm P3 Kategori ağacında sürekli rage click 7

Sonuç: tetikçi (issue açılması, Telegram alert) sadece ilk bir-iki insight için ateşleniyor, kalanlar arka planda birikiyor; aynı problem için "1 insight" diyince aslında 6'sı saklı kalıyor.

Kök sebep

apps/worker/src/lib/compress.ts:477 — session fingerprint inputs:

fingerprintHash([
  [...header.tags].sort().join(","),
  url,                          // ← /dashboard/vehicles/<UUID>/categories/<UUID>
  errors[0] ? normalizeError(errors[0]) : null,
  failedEndpoints[0] ?? null,
  header.severity,              // ← rage-click sayısına göre P1/P2/P3 değişiyor
]);

İki sorun:

  1. URL içinde UUID var — vehicleId ve categoryId her user için farklı → her session benzersiz fingerprint üretiyor → prisma.insight.findUnique({projectKey_fingerprint}) cache miss → yeni insight açılıyor.
  2. header.severity fingerprint'in parçası — aynı kök sebep, farklı rage-click yoğunluğu yüzünden bazen P1, bazen P2/P3 çıkıyor. Severity bucket'ın bir özelliği olmalı, kimliği değil.

Düzeltme

apps/worker/src/lib/compress.ts:

  • Yeni normalizePath() helper: UUID v1–v8, ULID, CUID2, sayısal id path segmentlerini :id ile değiştirir. Anlamlı path kelimelerine dokunmaz.
  • Yeni fingerprint inputs:
fingerprintHash([
  [...header.tags].sort().join(","),
  normalizePath(url),
  errors[0] ? normalizeError(errors[0]) : null,
  failedEndpoints[0] ? normalizePath(failedEndpoints[0]) : null,
]);

header.severity çıkarıldı. failedEndpoints da normalize ediliyor (örn. /api/vehicles/123/parts → /api/vehicles/:id/parts).

Tag set + normalized path + first-error + first-failed-endpoint yeterli ayrışım sağlıyor çünkü farklı UX problemleri zaten tagger'da farklı tag'ler alıyor (vin_decode_*, payment_*, search_validation_*, …).

Validation

apps/worker test runner kullanmıyor; smoke /tmp/check_fingerprint.ts ile yapıldı (12/12 ✓):

✓ normalizePath(/dashboard/vehicles/<UUID>/categories/<UUID>) = /dashboard/vehicles/:id/categories/:id
✓ normalizePath(/api/orders/12345/items)                     = /api/orders/:id/items
✓ normalizePath(/insights/i/cmpwrb82s002f14fza8lbc7f4)       = /insights/i/:id
✓ 6 kategori-bouncing URLs → 1 fingerprint(s) (expected 1)
✓ severity dropped from fingerprint (deterministic across calls)
✓ distinct tag sets stay split
✓ distinct errors stay split

pnpm -F worker typecheck temiz.

Etkisi

  • Sadece yeni compress'lenen session'ları etkiler. Historical insights satırları olduğu gibi kalır — eski yakın-duplicate insight'lar panel UI'dan elle merge edilmeli (founderNotes + status=dismissed).
  • İlk birkaç gün ek yan etki: bazı yeni session'lar artık eski insight'lara map'lenmek yerine, yeni doğru-dedupe edilmiş insight'lara map'lenecek; bu beklenen.
  • analyze.ts:79'daki prisma.insight.findUnique({projectKey_fingerprint}) cache lookup mantığı aynen çalışır.

Deploy

Merge → main → panel-worker otomatik redeploy.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

🤖 Generated with Claude Code

## ℹ️ Stack notu Bu PR `fix/insight-vin-validation-distinction` (PR #4) branch'i üzerine kuruldu çünkü PR #4 henüz merge edilmemişti. Sonuç: **PR #5 hem PR #4'ün düzeltmesini hem de fingerprint dedupe düzeltmesini içeriyor.** İki ayrı commit olarak görünür: 1. `9a479f9` — fix(insights): distinguish client-side VIN validation rejects from provider failures *(PR #4 ile aynı)* 2. `a7fe80f` — fix(insights): tighten session fingerprint so duplicate UX issues dedupe *(bu PR'ın asıl içeriği)* Yani **PR #5 merge edilirse PR #4'e gerek kalmaz** — PR #4 manuel olarak kapatılabilir. --- ## Sorun Aynı kök sebepli UX problemleri panel'de tek bir Insight olarak dedupe edilmiyor — son 1 günde **6 farklı `new` insight** açılmış, hepsi *"kategori ağacında bouncing → parts panel'i bulamama → rage click"* senaryosunu farklı kelimelerle anlatıyor: | id | severity | başlık | occ | |---|---|---|---| | `cmpclg7b9000dflajkg27hxvt` | P2 | Kategori ağacında rage click'ler → [sase.tr#76](https://git.semih.ai/root/sase.tr/issues/76) | 97 | | `cmp5yj1pa00071bakh9nqdh22` | P1 | VIN sonrası parça paneli sürekli yeniden yükleniyor | 34 | | `cmpwrb82s002f14fza8lbc7f4` | P1 | Parça arama filtresinde VIN girişi sonrası tıkanıklık | 1 | | `cmpwqgbyc002d14fzhnumfatw` | P2 | Kategori sayfasında tıklanabilir öğe bulunamıyor | 1 | | `cmpwlwt4a002314fzg8wyi3nh` | P2 | Kategori ağacında rage click + boş sayfa | 1 | | `cmpoeec4l001j14ozjjov5vrm` | P3 | Kategori ağacında sürekli rage click | 7 | Sonuç: tetikçi (issue açılması, Telegram alert) sadece ilk bir-iki insight için ateşleniyor, kalanlar arka planda birikiyor; aynı problem için "1 insight" diyince aslında 6'sı saklı kalıyor. ## Kök sebep `apps/worker/src/lib/compress.ts:477` — session fingerprint inputs: ```ts fingerprintHash([ [...header.tags].sort().join(","), url, // ← /dashboard/vehicles/<UUID>/categories/<UUID> errors[0] ? normalizeError(errors[0]) : null, failedEndpoints[0] ?? null, header.severity, // ← rage-click sayısına göre P1/P2/P3 değişiyor ]); ``` İki sorun: 1. **URL içinde UUID var** — `vehicleId` ve `categoryId` her user için farklı → her session benzersiz fingerprint üretiyor → `prisma.insight.findUnique({projectKey_fingerprint})` cache miss → yeni insight açılıyor. 2. **`header.severity` fingerprint'in parçası** — aynı kök sebep, farklı rage-click yoğunluğu yüzünden bazen P1, bazen P2/P3 çıkıyor. Severity bucket'ın bir *özelliği* olmalı, *kimliği* değil. ## Düzeltme `apps/worker/src/lib/compress.ts`: - **Yeni `normalizePath()` helper**: UUID v1–v8, ULID, CUID2, sayısal id path segmentlerini `:id` ile değiştirir. Anlamlı path kelimelerine dokunmaz. - **Yeni fingerprint inputs**: ```ts fingerprintHash([ [...header.tags].sort().join(","), normalizePath(url), errors[0] ? normalizeError(errors[0]) : null, failedEndpoints[0] ? normalizePath(failedEndpoints[0]) : null, ]); ``` `header.severity` çıkarıldı. `failedEndpoints` da normalize ediliyor (örn. `/api/vehicles/123/parts` → `/api/vehicles/:id/parts`). Tag set + normalized path + first-error + first-failed-endpoint yeterli ayrışım sağlıyor çünkü farklı UX problemleri zaten tagger'da farklı tag'ler alıyor (`vin_decode_*`, `payment_*`, `search_validation_*`, …). ## Validation `apps/worker` test runner kullanmıyor; smoke `/tmp/check_fingerprint.ts` ile yapıldı (12/12 ✓): ``` ✓ normalizePath(/dashboard/vehicles/<UUID>/categories/<UUID>) = /dashboard/vehicles/:id/categories/:id ✓ normalizePath(/api/orders/12345/items) = /api/orders/:id/items ✓ normalizePath(/insights/i/cmpwrb82s002f14fza8lbc7f4) = /insights/i/:id ✓ 6 kategori-bouncing URLs → 1 fingerprint(s) (expected 1) ✓ severity dropped from fingerprint (deterministic across calls) ✓ distinct tag sets stay split ✓ distinct errors stay split ``` `pnpm -F worker typecheck` temiz. ## Etkisi - **Sadece yeni compress'lenen session'ları etkiler.** Historical `insights` satırları olduğu gibi kalır — eski yakın-duplicate insight'lar panel UI'dan elle merge edilmeli (founderNotes + status=dismissed). - İlk birkaç gün ek yan etki: bazı yeni session'lar artık eski insight'lara map'lenmek yerine, yeni doğru-dedupe edilmiş insight'lara map'lenecek; bu beklenen. - `analyze.ts:79`'daki `prisma.insight.findUnique({projectKey_fingerprint})` cache lookup mantığı aynen çalışır. ## Deploy Merge → main → panel-worker otomatik redeploy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code)
root added 2 commits 2026-06-02 19:44:10 +03:00
fix(insights): distinguish client-side VIN validation rejects from provider failures 9a479f9a5b 
Reproducing insight cmpvfrjgc000114fzc7cdyh66 (a P1 "PL24 timeout" false
positive): trial user typed VW part numbers ("500 907 521", "5Q0 907 521")
into the VIN field on /; client-side regex rejected them with "Geçersiz şase
numarası. 17 karakter olmalı". No provider was called. The pipeline still
tagged the session as `vin_decode_fail_pattern`, routed to `provider_quality`,
and the LLM dutifully invented a PL24 outage.

Root cause spans three files:

1. tagger.ts grouped vin_decode_failed by `provider_attempted ?? source`. When
   `provider_attempted` is missing, `source: "landing"` (a UI location) was
   treated as a provider name, so a 1-provider set was synthesized and
   `vin_decode_fail_pattern` (P1) was emitted.

2. compress.ts formatCustom whitelist excluded `error`, `source`, `vin`. The
   LLM therefore never saw "Geçersiz şase numarası" or the offending input.
   Pattern 3 mechanical hypothesis told it "check provider health" regardless.

3. prompts.ts pickPromptTag routed any `vin_decode_fail_pattern` straight to
   `provider_quality` with no input-quality check, and the v3 system prompt
   had no guardrail for client-side validation rejects.

Fix:
- tagger: detect client-side rejects by `error` regex (Turkish + English) and
  by VIN shape (length != 17 or contains I/O/Q). When all fails are client
  rejects, emit new tag `vin_decode_client_validation_fail` at P3 instead of
  `vin_decode_fail_pattern` at P1. Real provider failures now require
  `provider_attempted` to be set (no more `source` fallback).
- compress: add `error`, `source`, `vin` to the formatCustom property
  whitelist so the LLM can see the actual failure context. Split Pattern 3
  into client-reject vs. real-provider-failure branches with distinct
  Turkish hypotheses.
- prompts: route `vin_decode_client_validation_fail` to `ux_friction` before
  the provider rule. Ship provider_quality v4 with an explicit guardrail
  instructing the model to return confidence ≤0.15 and reclassify when the
  inlined event properties show client-side rejection.

The seed-runtime upsert path deactivates the active v3 template on next
worker boot and inserts v4 in its place — no manual SQL needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix(insights): tighten session fingerprint so duplicate UX issues dedupe a7fe80f0c5 
Same root cause was producing one Insight row per user because URL carried
vehicleId/categoryId UUIDs and the raw URL fed the fingerprint hash. Result:
6 active "new" insights describing the same kategori-bouncing problem with
slightly different LLM phrasing, none deduplicated, only one (cmpclg7b9 →
sase.tr#76) had been triaged.

apps/worker/src/lib/compress.ts:
- new normalizePath() — collapses UUID / ULID / CUID2 / numeric path segments
  to ":id", conservative on plain words. Mirrors what the LLM already sees in
  the timeline.
- fingerprint inputs now:
    tags (sorted) | normalizePath(url) | normalizeError(errors[0]) | normalizePath(failedEndpoints[0])
- removed header.severity from the hash — severity is a property of the
  Insight bucket, not its identity; rage-click counts pushing the same root
  cause across P1/P2/P3 was forcing extra rows.

Smoke (12/12 pass via tsx /tmp/check_fingerprint.ts):
- 6 historical kategori-bouncing URLs → 1 fingerprint
- severity changes don't move the hash
- distinct tag sets / distinct errors still split

Historical rows are untouched — only new compressed sessions get the new
hash. Old near-duplicate insights can be merged manually via the panel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
root added 1 commit 2026-06-03 20:41:25 +03:00
feat(insights): semantic dedup gate + dismissed-theme suppression in analyze b61d955256 
The fingerprint hash (even after a7fe80f) still over-splits: the same root
problem produces different tag-sets / first-error / first-failed-endpoint
across sessions, so it hashes to a new fingerprint and creates a duplicate
insight. And generic ux_friction/frustrated_session sessions (category-tree
vs parts-panel vs schema-image) carry no structured signal to separate
sub-problems — only the LLM's reading of the timeline does. Dismissed insights
were only suppressed on an EXACT fingerprint repeat; a near-variant got a fresh
fingerprint and re-opened the theme. Dismissed rows were also excluded from the
fast-path cache, so every recurring session re-ran the LLM for no reason.

Two-layer dedup so (1) duplicates can't be created and (2) nothing similar to a
dismissed/duplicate theme is re-opened:

Layer 1 — fingerprint (or aliased fingerprint) fast-path, no LLM:
  - dismissed/duplicate  → SUPPRESS (bump occ/lastSeen, keep status)
  - validated/shipped    → flag REGRESSION (cheap, no re-analysis)
  - active & fresh        → attach occurrence
  - active & stale        → fall through to refresh re-analysis (unchanged)

Layer 2 — semantic dedup gate before creating a NEW insight (flash, no new infra):
  - build a bounded catalog: all active insights + recently-touched
    dismissed/duplicate/validated/shipped (lookback-windowed, capped)
  - ask the model whether the session is the SAME underlying problem as a
    catalog entry (anti-hallucination: only accept catalog ids; conf >= 0.72)
  - on match: alias this fingerprint onto that insight + attach/suppress, so the
    next identical session fast-paths in Layer 1 (no LLM, no duplicate row)
  - no match: create a new insight as before

Schema: Insight.aliasFingerprints String[] (additive; applied via prisma db push
on deploy). Dedup-gate cost is logged to costLedger as promptTag=dedup_gate.

Tunables: INSIGHT_DEDUP_GATE (default on), INSIGHT_DEDUP_THRESHOLD (0.72),
INSIGHT_DEDUP_SUPPRESS_DAYS (90), INSIGHT_DEDUP_CATALOG_MAX (120).

Pure decision helpers (classifyExistingAction, shouldAcceptMatch) extracted and
covered by a smoke test (apps/worker `pnpm test`, 18/18). tsc --noEmit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
root merged commit 1a95a8823f into main 2026-06-03 20:47:12 +03:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
root
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: root/sp#5
Delete Branch "fix/insight-fingerprint-dedupe"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?