Fiat (fiatp_parts/fiatt_parts) was misconfigured as LEGACY_FIAT basePath /fca,
which 404s on every request — PL24 Fiat decode was dead. Live discovery
(de-708171) shows Fiat is a standard P5 Modern catalog at /p5fiat: directAccess
+ maingroups/subgroups/parts/images all match the existing P5 flow. Only the
vinfoBasic record shape differs ({key,description} vs {values:{description,value}}).
- types: fiatp_parts/fiatt_parts -> P5_MODERN, apiPath/basePath /p5fiat
- parseVehicleResponse: parse the p5fiat vinfoBasic shape; friendly model from
"Model bilgisi"; year from MY / production date
Covers European (ZFA) Fiats + some commercial Tofas (fiatt). Turkish Tofas
passenger VINs (NM4, incl. Egea) are not in this catalog. de account separation
(resolveAccount Rule 1) unchanged; de auth handshake proxied, catalog data not.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lands the user-facing half of the unsubscribe preferences work. The
one-click endpoint already shipped in this PR's main commit; this adds
the proactive self-service path at /dashboard/settings?tab=notifications
so users don't have to wait for a mail to land before tuning their
preferences.
Backend
-------
New EmailPreferencesController at /api/email/preferences:
GET → returns one row per OPTIONAL_WORKFLOWS entry, each with current
optedOut boolean (false when no DB row exists).
POST → body {workflow, optedOut} flips the row; source='settings_page'
captured for the audit trail.
Auth+payment workflows are deliberately not exposed — the server's
OPTIONAL_WORKFLOWS set stays the single source of truth.
Frontend
--------
Adds a 'notifications' tab to /dashboard/settings (between 'preferences'
and 'security'). One toggle row per optional workflow with TR copy that
explains what each mail is for. Optimistic update — switch flips
instantly and reverts on failure; PostHog event captures accept/reject.
Static footer note clarifies that auth + payment mail keeps coming
regardless of the switches above (so users don't think they've
unsubscribed from password-reset).
i18n
----
Added settings.tabs.notifications + settings.notifications.{title,
description} to both tr.json and en.json. Body copy is hard-coded TR
(matches audit §9.3 #11 TR-only decision).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lands the §9.3 "compliance + brand" tier of mailAudit.md as one PR. Six
changes share enough surface (notifications, shared utils, infrastructure)
that splitting them would require multiple stacked PRs.
#9 — Turkish-locale title-case for names at signup
• New `normalizeName()` in @sase/shared, locale-aware (İ/ı pairs handled
via toLocaleLowerCase('tr-TR') + matching toLocaleUpperCase). Hyphen-
aware, collapses whitespace, idempotent.
• Wired into better-auth's `user.create.before` hook so every new signup
gets canonicalised before the row lands in Postgres.
• 28 unit tests in packages/shared/src/index.spec.ts.
• Backfill script at `scripts/backfill-user-names.ts` (already run
against prod + dev — 210/402 prod users and 72/153 dev users
canonicalised, plus 71 Novu subscribers).
#10 — Email typo correction at signup
• New `suggestEmailFix()` in @sase/shared: exact-match typo dictionary
for the addresses we've actually suppressed (icould.com, gmial.com,
xn--gmail-bgd.com, …) plus Levenshtein ≤ 2 fallback against popular
providers.
• Inline UI hint on the register form — "Bunu mu demek istedin? <link>"
that swaps the email on click; PostHog event tracks acceptance.
#11 — Strip EN branches (decision: TR-only)
• 0/205 prod subscribers have locale='en' and there's no marketing in
English — the {{#equals subscriber.locale "en"}}…{{else}}…{{/equals}}
framework was dead code in all 10 templates.
• Templates updated in-place (avg ~30 % smaller). Renamed
`novu-welcome-tr.html` → `novu-welcome.html` for consistency with the
other 9 files.
• Novu workflow definitions in both Dev + Prod envs updated via Mongo:
subjects collapsed to TR-only, content replaced with new HTML
(mongodump/restore-safe).
• App code: `NovuRecipient.locale` and `NovuUser.locale` removed; the
`...(user.locale === "en" ? { locale: "en" } : {})` spread in NovuService
is gone.
#12 — DKIM rotated to 2048-bit RSA
• Postal default was 1024-bit (selector `postal-YeIm3w`). Generated new
2048-bit key, added DNS TXT `postal-2k260604._domainkey.sase.tr`,
atomically swapped `domains.dkim_identifier_string` +
`dkim_private_key` in Postal MariaDB, restarted Postal SMTP.
• Verified: outgoing welcome mail now signs with `s=postal-2k260604`
and a 256-byte signature body (vs the previous 128-byte 1024-bit
signature). Pubkey on DNS matches the private key.
• OLD TXT record (`postal-YeIm3w._domainkey`) stays in DNS for ~7 days
as a grace window for in-flight mail.
#13 — MTA-STS + TLS-RPT
• Extended the existing mailtrack Cloudflare Worker to also serve
`mta-sts.sase.tr/.well-known/mta-sts.txt` (`mode: enforce, mx:
mx.postal.sase.tr, max_age: 604800`). Workers Domain bound to the
mailtrack service via Cloudflare API.
• DNS:
`_mta-sts.sase.tr` TXT "v=STSv1; id=20260604111347"
`_smtp._tls.sase.tr` TXT "v=TLSRPTv1; rua=mailto:dmarc@sase.tr"
• Verified policy fetch returns 200 with the expected body; cert valid
(sase.tr SAN issued by GTS).
#14 — Unsubscribe preferences + RFC 8058 one-click endpoint
• New `email_preferences` table (migration 0011) keyed
(user_id, workflow), captures source for audit
(one_click / manual_link / settings_page).
• New `UnsubscribeController` at `/api/email/unsubscribe`:
- POST: Gmail/Yahoo one-click bot path (200 fast)
- GET: human-visit, renders a Turkish confirmation page
Both validate an HMAC-SHA256(`userId|workflow`) token under
`UNSUBSCRIBE_SECRET` — stateless, no DB lookup to validate, secret
rotation invalidates all outstanding tokens.
• `triggerNovu()` now mints the per-call `overrides.email.headers`:
`List-Unsubscribe: <https://…?u=&w=&t=>, <mailto:unsubscribe@…>`
`List-Unsubscribe-Post: List-Unsubscribe=One-Click`
Auth + payment workflows opt out via NO_UNSUBSCRIBE_WORKFLOWS so the
unsubscribe URL never appears on transactional mail.
• `NovuService.trigger()` pre-flight-checks `isOptedOut()` and skips the
trigger entirely if the user opted out. Fail-open on DB error so a
transient blip can't swallow auth mail.
• `lifecycle-email.processor.ts` (standalone BullMQ worker — no NestJS
DI) does the same check inline via a LEFT JOIN on
`email_preferences WHERE opted_out IS NULL`.
• Coolify env wired in both Prod and Dev apps:
`UNSUBSCRIBE_SECRET` (32-byte hex, distinct per env)
`UNSUBSCRIBE_URL_BASE` = `https://(dev.)sase.tr/api/email/unsubscribe`
## Companion sibling changes (already applied, NOT in this PR)
- Cloudflare worker `mailtrack` redeployed with mta-sts.sase.tr custom domain.
- Postal MariaDB `domains.dkim_identifier_string` + `dkim_private_key`
updated to the new 2k260604 selector (live since 2026-06-04 11:18).
- `postal-2k260604._domainkey.sase.tr` TXT record live at Cloudflare.
- `_mta-sts.sase.tr` + `_smtp._tls.sase.tr` TXT records live at Cloudflare.
- Novu Mongo notification + message templates updated to TR-only.
- 282 user names canonicalised across prod + dev + Novu subscribers.
## Verification snapshot
- Postal raw_headers (ID 157, post-rotation): `s=postal-2k260604` + 256-byte b=
- `dig +short TXT _mta-sts.sase.tr @1.1.1.1` ⇒ live id=20260604111347
- `curl https://mta-sts.sase.tr/.well-known/mta-sts.txt` ⇒ 200 with policy
- 28 unit tests (normalizeName + suggestEmailFix) all green via Node sanity.
## Deploy notes
- Re-run `pnpm db:generate` to regenerate the drizzle snapshot for 0011
(added the journal entry manually because no drizzle-kit on this box).
- Run `pnpm tsx scripts/backfill-user-names.ts --apply` against any DB not
yet canonicalised (already done for prod + dev today).
- The host-side Novu nodemailer-headers patch at
`postal/novu-patches/apply-headers-patch.sh` must be re-run after every
Novu container redeploy or the List-Unsubscribe header is silently dropped
before reaching Postal (see audit §9.1 #3 for the upstream cause).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Same shared-core + thin-brand pattern (P4BrandHooks). Both decoded with polluted models (Opel kept
the platform code "…: P10"; Hyundai kept the VIN breadcrumb), wrong VIN-char years, empty transmission.
Fix reads the vin-group info grid + title:
- Opel: model from <title> (segment after the platform code), year from Model yili, transmission from
Şanzıman kodu, engine from Motor tipi. Verified 6/6 prod (ASTRA-J/2014, CORSA-D/2011, INSIGNIA-A, AGILA-A…).
- Hyundai/Kia: model from <title>, build-year from Üretim tarihi, transmission/body from the
ENGLISH-labelled grid (plain lowercase — tr-locale would map I→ı and miss "TRANSMISSION").
Verified (GETZ 02/2004/5 SPEED MT, RIO / STONIC 17/2019…).
Nissan deferred: both prod Nissan VINs are unresolvable upstream (PL24 returns "Model seçimi", no
model) — nothing to parse. Engine/categories/drill unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ford decoded with a polluted model ("Ford Nutzfahrzeuge {VIN}: Transit Connect - TC7…", from the
page title), a wrong VIN-char year, and empty transmission. The clean data is in the vin-group info
grid (Araç Hattı=model line, Üretim tarihi=build date, Vites Kutusu=transmission, Motor Tipi=engine,
Gövde Tarzı=body). New thin PL24FordService supplies parseFordVinInfo via the shared P4 brand hook;
orchestrator routes LEGACY_FORD to it. Engine/categories/drill unchanged.
Verified live vs 9 prod Ford VINs: 7 now decode clean model + correct build-year + transmission
(Transit Connect/2006, Mondeo/1997 CD4E Otomatik, Focus/2000, …) — VIN-char years were off by up to a
decade and transmission was empty for all. 2 VINs are upstream gaps (no Araç Hattı → unchanged).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Volvo decoded as "Volvo {VIN}" with no real model: the generic Ford-shaped
parseP4VehicleResponse reads window.vehicles/<title>/<h1>, but Volvo ships model/year/type in a
<table id="vinInfoTable"> caption/value grid (Model="V60 Cross Country (19-)", Model yili=2021,
Türü="V60 CC II"). 9 vehicles affected (3 dev + 6 prod).
- Add a per-brand vehicle-info hook (P4BrandHooks.parseVehicleInfo) to the shared P4 engine
(PL24FordLegacyService); brand values win, generic fills gaps. Backward-compatible: no hook → identical.
- New thin PL24VolvoService supplies parseVolvoVinInfo (vinInfoTable parser); orchestrator routes
LEGACY_VOLVO decode to it. Categories/drill unchanged (shared engine).
Verified live vs 6 prod Volvo VINs: all decode real models (S80/S60/S40/V40/EX40·XC40/V60 CC) +
correct years; drill intact (motor → 7 subgroups). First step of the per-brand split
(shared core + thin brand services).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dev.sase.tr was running the daily lifecycle-email cron (trial-ending,
win-back) against sase_dev, which is a seeded copy of the prod DB —
real customers were getting duplicate mails (one from prod, one from
dev). Mirror the isCatalogBackfillEnabled() gate so the lifecycle cron
only registers on the canonical prod host (COOLIFY_FQDN=sase.tr or
BETTER_AUTH_URL=https://sase.tr). LIFECYCLE_EMAIL_ENABLED env can force-
enable for staging testing. The else branch removes any stale scheduler
from Redis so a previously-registered cron stops firing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The trial-ending / payment-success / payment-failed emails were pointing
to /dashboard/settings?tab=subscription which does not exist; the correct
path is /dashboard/subscription. Referral paths are unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PSA (Peugeot/Citroën/DS) VIN decode was systemically broken: the catalog
vin-group page returns all families unfiltered, so decode fell back to the first
family/salesType (a manual base variant) — yielding "{Brand} {VIN}" model names,
empty transmission, wrong model year, and manual-only parts trees (automatic
gearbox parts missing). Reported for a 1999 Peugeot 106 automatic shown as a 2029
manual with no automatic parts.
- New self-contained PL24PsaService: consumes PL24's FI flow (vin.action →
hintstoken → FI page → json-vin-main-groups → json-vin-illustrations →
vin-image-board). Reads model/year/transmission from the FI identification
table; builds the VIN-indexed parts tree (correct per actual VIN). Does not
touch Ford/Volvo/Nissan/Opel/Hyundai-Kia/Fiat.
- Orchestrator + categories.service route PSA VIN decode/drill to the new service.
- Cycle-correct extractModelYear in @sase/shared (X→1999, not 2029): resolve the
30-yr VIN year code to the most-recent plausible year (≤ now+1); dedupe 6 copies.
Validated live against 13 already-decoded PSA VINs: 12/13 full trees with real
model/year/transmission; automatics correctly detected (106 BVA, 206 AL4, 3008 BVA8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Second batch from undecoded-vin-rca.md. Every entry verified (authoritative
WMI registry + prod sibling + VDS fingerprint + live PL24 probe) before adding
— a wrong brand label or a dead PL24 route both have real downside.
Q5 — WMI_BRAND_MAP (packages/shared, UX/telemetry only; does NOT change which
catalog decodes). Adds 9: W1V=Mercedes-Benz, VXF=Fiat, YAR=Toyota, NL1=Hyundai,
KPA=SsangYong, PL1=Proton, LSV=Volkswagen, LVV=Chery, ZCF=Iveco. Resolves the
RCA conflicts: VXF=Fiat (not Opel), YAR=Toyota ProAce (not Opel); NL1=Hyundai
(not TOGG — MY char predates TOGG). Deliberately NOT added: WF1/WAA/W0W (1-char
corruptions of WF0/WAU/WVW — mapping them would mislabel typo classes), VE1
(unresolved, deferred).
Q6 — PL24_WMI_SERVICE_MAP (real decode routing). Both target services are
proven live, so these only append WMIs to working routes (no dead-route →
isDecodeable()=true → 25s-budget/circuit-breaker hazard):
- SJN/VSK/MNT → nissan_parts (Sunderland/Spain/Thailand) — regression restore,
dropped in the 757905f refactor; nissan_parts proven (JN1 decodes in prod).
- W1V → mercedesvans_parts (Sprinter/Vito) — retires the RCA "never tried" flag:
mercedesvans_parts decodes WDF44770513186937 live today via the same P5 path.
Tests: shared getBrandFromWmi (adds + corruption-skips return null), pl24 route
map. Full api suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Second batch from undecoded-vin-rca.md. Every entry verified (authoritative
WMI registry + prod sibling + VDS fingerprint + live PL24 probe) before adding
— a wrong brand label or a dead PL24 route both have real downside.
Q5 — WMI_BRAND_MAP (packages/shared, UX/telemetry only; does NOT change which
catalog decodes). Adds 9: W1V=Mercedes-Benz, VXF=Fiat, YAR=Toyota, NL1=Hyundai,
KPA=SsangYong, PL1=Proton, LSV=Volkswagen, LVV=Chery, ZCF=Iveco. Resolves the
RCA conflicts: VXF=Fiat (not Opel), YAR=Toyota ProAce (not Opel); NL1=Hyundai
(not TOGG — MY char predates TOGG). Deliberately NOT added: WF1/WAA/W0W (1-char
corruptions of WF0/WAU/WVW — mapping them would mislabel typo classes), VE1
(unresolved, deferred).
Q6 — PL24_WMI_SERVICE_MAP (real decode routing). Both target services are
proven live, so these only append WMIs to working routes (no dead-route →
isDecodeable()=true → 25s-budget/circuit-breaker hazard):
- SJN/VSK/MNT → nissan_parts (Sunderland/Spain/Thailand) — regression restore,
dropped in the 757905f refactor; nissan_parts proven (JN1 decodes in prod).
- W1V → mercedesvans_parts (Sprinter/Vito) — retires the RCA "never tried" flag:
mercedesvans_parts decodes WDF44770513186937 live today via the same P5 path.
Tests: shared getBrandFromWmi (adds + corruption-skips return null), pl24 route
map. Full api suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ConfigService.get<number>("EMEX_PROXY_PORT_START") returns the raw env
STRING; the port-pick arithmetic then string-concatenated it
(45 + "10001" = "4510001"), producing an out-of-range port that made
undici's `new URL` throw "Invalid URL" at EmexService construction —
crashing the entire API on boot.
A single-port range (823) happened to concat to a still-parseable "0823",
which masked the bug for months. It surfaced the moment the prod
EMEX_PROXY_PORT range was widened (823 -> 10001-10099) to let Q1's
per-request port rotation work: prod crash-looped until the env was
reverted. Coerce to a validated integer port (1-65535) with default
fallback so a real range is safe.
Regression test: constructing EmexService with string port env over a
real range must not throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-cause analysis live re-decoded all 124 historically-undecoded prod
VINs; 38 already decode now. These 4 fixes target confirmed code bugs
that drop or mask real decodes (see undecoded-vin-rca.md):
Q2 — PL24 circuit breaker now only counts transient transport faults. A
definitive upstream negative (NotFound/BadRequest) no longer trips the
global 30s breaker that was starving PL24 for every subsequent VIN
(the sibling-VIN inconsistency in the report). Live-proven on VR7.
Q3 — previewVin / multi-candidate path no longer returns an empty
success: the pcat/emex candidate branches fill brandName (from catalogId
/ WMI), fixing the 6 "HTTP 200 with null brand+model" cases.
Q1 — EMEX fetch retries transient proxy failures with a FRESH ProxyAgent
per attempt (rotates the DataImpulse port; ~42% blip rate observed),
plus an opt-in direct fallback (EMEX_DIRECT_FALLBACK). HTTP answers are
never retried.
Q4 — VIN resolve cache keys namespaced by DECODE_CHAIN_VERSION and the
negative TTL drops 6h -> 30m, so a decode-chain fix self-heals stale
negatives on deploy instead of masking phantom-undecoded VINs for hours.
The admin cache-buster uses the same key builder.
Tests: 179 passed (+ new Q2/Q3/Q4 specs). typecheck + biome clean.
Deploy note: prod EMEX_PROXY_PORT_START/END are both 823 (single port);
widen to a real range (e.g. 10001-10099) in Coolify so Q1's port
rotation takes full effect.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase-1 of the backfill scan selects all zero-parts decoded vehicles every wave.
Genuine residue (VINs with no catalog data anywhere — model-indexed HKN,
EMEX-uncovered, etc.) stays zero-parts forever, so it filled the batch every
wave, re-attempting dead vehicles and starving the Phase-2 rolling rescan (its
cursor was stuck for a week).
Track a per-vehicle no-result counter (prefetch:noresult:<id>) incremented when a
prefetch attempt finishes with the vehicle still at zero parts (0 categories in
init, or 0 parts after the whole chain). tryPick skips vehicles past
PREFETCH_NORESULT_MAX (default 2) attempts; the counter has a TTL
(PREFETCH_NORESULT_TTL_DAYS, default 7) so a later catalog fix re-fills them.
Frees capacity for fillable vehicles and lets Phase-2 run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After Tier 1 removed the self-throttle, the 5 jobs/min limiter + concurrency 1
became the bottleneck. Make throughput env-tunable so prod can ramp while
watching the fail rate:
- concurrency 1 -> 3 (PREFETCH_CONCURRENCY): parallelises emex/pl24 so a slow
parts-catalogs job no longer head-of-line-blocks the queue.
- rate ceiling 5 -> 20 jobs/min (PREFETCH_RATE_MAX).
- parts-catalogs: drop the pathological cumulative index*20s enqueue delay (the
Nth leaf of a vehicle waited N*20s); keep one per-job pace (PCAT_PACE_MS,
default 15s, 0 to disable).
- PL24 09-18 scrape window now env-tunable (PREFETCH_PL24_START / _END).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The worker's own upstream fetches called touchActivity(), setting the
prefetch:activity:<source> cooldown key (TTL 300s) that checkCooldown then
honoured — so after each fetch the worker paused itself for up to ~5 minutes
(re-checking every 60s, ~5 empty cycles per key). At ~1 fetch / 5 min the
3387-job backlog needed ~6 days to drain.
- Wrap each worker job in an AsyncLocalStorage backfill context; touchActivity
skips the cooldown key when invoked from the worker, so the cooldown reflects
only real user requests (worker yields to users, never to itself).
- Cooldown TTL 300s -> 90s (a request 5 min ago isn't "active").
- checkCooldown pauses for the key's actual remaining TTL (one wait) instead of
a fixed 60s re-check loop.
No extra upstream load — only removes the worker's self-imposed idle time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No PostHog event carried realized revenue, and EFT/havale activations fired
nothing at all — so total paid revenue / MRR was unmeasurable (a Stripe DWH
connector alone would also miss EFT). activateSubscription is the shared
chokepoint for both Stripe (stripe.service) and EFT/manual (billing.service)
activation, so emit one canonical subscription_activated there with PostHog
revenue props: $revenue (major TRY), currency, mrr (yearly amortised /12),
plan, plan_id, brand_count, billing_period, method (looked up from the latest
payment row), referral_credit_days. Funnel steps keep their kuruş 'amount' but
intentionally carry no $revenue, so revenue isn't double-counted.
Unblocks trial->paid, MRR/ARPU and revenue-by-plan/channel across ALL payment
methods. Injected PostHogService (PostHogModule is @Global).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PL24 sub-group nav nodes were classified by linkWid.includes("Group")
(case-sensitive). That matched capitalised wids (subGroupsTable) but
missed lowercase ones — groupReferenceTable, groupTable, groupsTable
(~1157 leaf nodes in prod) — so those skipped the group-drill branch in
getCategoryWithPartsInner and the reference-resolution descent, falling
to the parts path (a wasted upstream fetch; the generic drill-on-empty
fallback then re-drilled them). Lowercasing the check routes these nav
nodes straight to children/loadError like their capitalised siblings.
Empty-catalog audit (2026-06-03) showed PL24 drives 61% of user-seen
'0 parça' views; pcat fake-leaves are effectively solved (1 case).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PL24 translates "see table" two ways — "bk. tablo:" and "bakınız tablo,
konum:". Detection only matched the first, so the latter rows (e.g. evaporator
housing → 820-020) stayed dead. Broaden the name regex to match either, and
strip both phrasings from the displayed label. The code-in-remark gate still
prevents flagging real parts that merely mention "tablo".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a reference's target illustration isn't seeded yet (load-time index
miss → categoryId null), clicking it now calls a new resolve endpoint that
drills the relevant main-group root (its external_id = the code's first
digit; the illustration is a direct child) and re-resolves. One PL24 call in
the common case, bounded + cached; falls back to pre-filled search if not
found. UI shows a spinner on the button while drilling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PL24 BOM emits "see table NNN-NNN" reference rows (oem N/A, target code in
remark) with NO upstream link. Resolve the code against the vehicle's
illustration index (codes live in category names as {NNN-NNN}) and render
jump links. Unresolved targets (branch not seeded yet) deep-link a pre-filled
catalog search via ?q=.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Volvo's vin-image-board.action ships its BOM inline as partno= tc-data-row
rows (no pncHierCode / json-vin-bom-detail), so the Ford VIN-BOM parser
returned 0 parts. Fall back to parsePsaBomParts (partno= rows) when no pnc
rows are found. Also decode HTML entities (Ö " …) in scraped Volvo
subgroup names. Completes the Volvo chain: group1→group2→illustration→parts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bugs in the vin-group.action subgroup scrape: (1) the filter dropped any
href containing "openVinDialog", but the real sub-group links carry
openVinDialog=false (only the VIN-dialog crumb is =true) — so every child was
discarded; (2) the deepest group level lists its illustration leaves as
vin-image-board.action links, which weren't extracted. Now match both deeper
vin-group.action?groupN= and vin-image-board.action anchors, and only drop the
openVinDialog=true crumb. Completes Volvo group1→group2→illustration→parts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PL24 serves a stripped NOT_LOGGED_IN_DEMO page (no groups/parts) when the
service token is stale. decodeVinForService retries on this, but drill paths
(fetchSubGroupsByPath/fetchPartsByPath) reach upstream only via fetchP4Page,
which didn't — so Volvo subgroup drilling parsed empty demo pages. Retry once
with fresh auth on a demo page. Adds a Volvo-drill diagnostic log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Volvo vin-group.action categories store hrefs relative to the catalog dir
(e.g. "vin-group.action?group1=…"). fetchP4Page did `${baseUrl}${url}`,
collapsing to "partslink24.comvin-group.action" → ENOTFOUND. Prefix the
service basePath when the path is relative. Fixes Volvo subgroup drilling
for existing (relative) stored linkPaths without a re-decode.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backfill + reactive prefetch can re-drill the same category multiple
times, and the parts insert path had no dedupe guard. Result: 8.2%
duplicate rows on pl24, 14.5% on parts-catalogs, and 33.7% on emex —
~108k extra rows across 7,571 categories on 224 vehicles. Every drilled
catalog page rendered each part twice (the Tampon example: 32 rows for
19 distinct OEMs).
* Migration 0010 — phase 1 deletes existing dupes preserving the
oldest row per (vehicle_id, category_id, oem_code, name, position)
group; phase 2 adds a UNIQUE INDEX over the same tuple with NULLS NOT
DISTINCT (PG 15+) so null position/vehicle_id collapse like equal
values rather than each counting as its own "distinct" row. Idempotent
CREATE UNIQUE INDEX IF NOT EXISTS so the runner is safe to re-apply.
* All five insert(parts).values(...).returning() call sites
(parts.service, categories.service ×3, catalog.service) get
.onConflictDoNothing() so future re-drills no-op instead of erroring
on the new constraint. `.returning()` continues to surface only the
newly-inserted rows; existing logs read `Stored N parts` as actual
net insertions, which is what we want.
Dry-run on dev DB: 524,540 → 425,186 parts (99,354 dupes deleted), index
created cleanly. Same delta expected on prod (~108k drop).
drizzle-orm 0.41 doesn't expose .nullsNotDistinct() on the index builder
so the constraint is owned by raw SQL — see the inline comment in the
parts schema and the migration file. Future schema generators should NOT
try to drop or rewrite this index.
Volvo's VIN catalog is a 3-level vin-group.action?group1=…[&group2=…] HTML
tree; PL24's json-vin-*-group.action JSON endpoints now 404. Add HTML
subgroup extraction (keep links one group-level deeper than the current
path) in ford-legacy fetchSubGroupsByPath, and route vin-group.action?group1=
nodes through getChildren (drill-first, fall back to parts) in
getCategoryWithParts. Recovers Volvo group1→group2 navigation. Leaf parts
handled separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backfill was sweeping only 20 vehicles/hour and skipping pcat outside
09:00–19:00 Istanbul, leaving the catalog backlog (115 pl24 + 63 pcat +
27 emex zero-parts vehicles) crawling forward at ~528 parts/day and the
biggest user vehicle (Toyota Corolla 2026) unchanged across 24h.
Two related changes — both unblocked by the Redis-persisted warm JWT pool
(aa4d055 + dcf7e06) which keeps pcat captures alive 24/7 instead of
needing the 09:00–19:00 office-hours assumption:
* isWithinTimeWindow: parts-catalogs no longer gated — the warm pool +
Redis hydration cover the cold-start case the old office-hours rule
was working around. PL24 keeps its 09:00–18:00 window because the
upstream rate-limit is still tighter outside it.
* BACKFILL_BATCH_SIZE 20 → 40 — twice as many vehicles per wave, still
protected by MAX_BACKLOG=1000 self-throttle and per-source cooldown
(prefetch:activity:<source>) so live-user traffic still gets priority.
Combined effect: pcat goes from ~10h/day to 24h/day, batch doubles —
roughly 3× backfill throughput. Worst-case proxy spend tracked by the
DataImpulse daily cap; if a wave saturates upstream the rate-limit
handler (c7e59b9) defers without burning attempts.
Volvo (VIN-indexed legacy catalog) ships its real top groups as
vin-group.action?group1=… in the vin-group HTML. PL24's
json-vin-main-group.action endpoint now 404s, so decode falls back to the
HTML scrape — but parseP4NavigationCategories AND the seed-time NAV_CRUMB
filter both blanket-exclude vin-group.action, dropping every real Volvo
group → 0 categories. Exclude vin-group.action as a crumb only when it
lacks group1=. Confirmed upstream: 7 real groups (Frenler, Elektrik
sistemi, …) present in the HTML for YV1AS7050A1118639.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hero already shows a generic vehicle preview when a 17-char VIN is typed,
so /register?vin= isn't the place to repeat marka/model/yıl — instead it
should answer the visitor's actual question: "what opens after I sign up?"
Adds a public catalog-stats endpoint and a data-driven teaser card on the
register page:
Backend:
* GET /api/vehicles/:vin/teaser-stats (Public, VIN-validated). Single SQL
round-trip counts categories + parts + schema_pics for the VIN. Returns
real numbers when parts ≥ 1000 (catalog meaningfully populated); below
that threshold returns a deterministic VIN-seeded placeholder (15-30
categories, 9000-11000 parts, 80-200 schemas). Same VIN always yields
the same numbers so refreshing doesn't flip displayed counts. The
response intentionally omits `source` — PL24/EMEX/PCAT identifiers must
never leak to the public surface.
Frontend (/register):
* When ?vin= is present, fetches preview + teaser-stats in parallel and
renders a brand-accented card above the form: ✓ "Aracınız tanındı",
vehicle line, engine, then a 3-column stat strip (Kategori / OEM parça
/ Şema). Below: "Hesap açtığında bu araç için kataloğa anında erişim
açılır."
* B2B copy pass on the rest of the page:
- Heading flips to "Hesap Aç ve Katalogu Gör" when VIN present
- Trial messaging rewritten to anti-gimmick B2B tone:
"Kart bilgisi gerekmez · 30 gün ücretsiz · istediğin an iptal"
(was: "30 gün Full Paket ücretsiz deneyin — kredi kartı gerekmez")
- Subhead: "Sınırsız şase sorgulamak için ücretsiz hesap aç"
- Submit button: "Hesap Aç ve Katalogu Gör" (vin) / "Hesap Aç" (no vin)
- "Ücretsiz Başla" / "Full Paket" strings purged per [[sase-b2b-copy-not-consumer]]
isP4LegacyPath matched any ".action" path, shadowing the PSA
json-illustrations.action / image-board.action dispatch in
fetchSubGroupsByPath and fetchPartsByPath. Every PSA (Citroen/Peugeot/DS)
drill below main-group level fell through to the Ford/Fiat legacy parser,
which cannot parse PSA JSON, so it returned empty. PSA vehicles decoded
since the catalog module landed (72c0de6) showed categories but 0 parts
(41/43 affected). Exclude /psa/ from isP4LegacyPath so these paths reach
fetchPsaIllustrations / fetchPsaParts. Heals existing vehicles on demand;
no re-decode needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the old marketing "guided tour" /demo with a real, fully-functional
catalog browsing experience for the pre-warmed example vehicle. No auth
required, no upstream calls — entirely served from prod DB.
Backend (apps/api/src/demo):
* New @Public() controller exposing five endpoints under /api/demo:
- GET /vehicle → demo vehicle metadata
- GET /categories/tree → top-level category tree
- GET /categories/search?q= → cross-tree search
- GET /categories/:id → getCategoryWithParts (parts+schema+hotspots)
- GET /categories/:id/children → drill children
* DemoService validates every category id against DEMO_VEHICLE_ID before any
downstream service call — the public surface can't be used to read an
arbitrary vehicle's catalog (1-row SELECT, NotFound on miss or wrong owner).
* Vehicle id is env-driven (DEMO_VEHICLE_ID, defaults to the pre-warmed
WVWZZZ1JZ3W597935 — VW Golf 2003 with 277 cats / 9841 parts / 178 schemas
fully drilled in prod).
* Wires CategoriesModule (already exports CategoriesService) — zero new
business logic, just a thin public façade.
Frontend (apps/web):
* /demo (replaces old marketing page): vehicle header + top categories grid
reading /api/demo/* + sticky DemoBanner with sign-up CTA.
* /demo/categories/$categoryId: drill page rendering either a children grid
(parent) or the existing SchemaViewer + parts panel (leaf) — same shape
the dashboard uses, so hotspot overlay, breadcrumb trail, retry on
upstream loadError all just work.
* DemoBanner: sticky top, "Örnek araç: {label} — Kayıt Ol" CTA. The
"Yeni VIN sorgula" explicit paywall trigger lands in a follow-up task.
* PostHog events: demo_loaded (source query-param-aware),
demo_category_clicked, demo_category_detail_viewed, demo_to_register_click
(banner / footer / category_footer placements).
* usePageMeta gains an opt-in `noindex` flag — demo sets it to noindex,follow
for the first 4-6 weeks per spec; cleaned up on unmount so SPA navigation
doesn't carry it to the next route.
The post-goto token poll ran a blind 20s wait regardless of whether
page.goto succeeded or threw. On a healthy goto the widget API call
fires within ~1-2s; on a failed goto the request either already went
through (rare) or never will (common). The 20s cap was the dominant
cost on failed-site attempts — verified tonight as a 28s "No token
after ..." log on auto-komplekt after page.goto ERR_TIMED_OUT.
* CAPTURE_POLL_AFTER_OK = 10 (5s) — token usually arrives in <2s
* CAPTURE_POLL_AFTER_FAIL = 4 (2s) — brief grace then bail
Per-attempt worst case on a dead site: 10s goto + 2s grace = 12s
(was 10s + 20s = 30s). On a healthy site, well-known capture times
(3-5s) stay comfortably inside the 5s post-goto cap.
A captured slot now lives in Redis under `pcat:jwt:slot` for the JWT's
remaining lifetime (minus a 60s safety buffer). On module init we try
Redis before launching Playwright — if a fresh slot is there we adopt it
and schedule its refresh, skipping the ~5s capture entirely. After every
successful capture+validation we publish to Redis so the next restart (or
any sibling pod) can inherit. invalidateSession deletes the Redis copy
because a 401/403 means the cached IP-binding is dead.
Token is still IP-bound to its proxyPort. If a hydrating container reads
the slot but the proxy has rotated away from the captured IP, the next
upstream call 401s and the existing invalidateSession fallback re-captures
locally — so worst case = today's cold-capture behavior, never worse.
Note: dev and prod use separate Redis instances. This patch reaches PL24
parity (same-env redeploy hydration); a true dev↔prod shared cache would
need either an external Redis or an internal-token bridge.
Concurrent captures are bound by how many distinct partner sites we can
drive in parallel — each capture needs its own site so they don't collide
on the synchronous siteIndex round-robin. Ports are effectively unlimited
(10000-10999) and Playwright contexts are isolated, so the previous
arbitrary cap of 3 left headroom on the table when the pool is cold and
N>3 user clicks race in at once.
Tying the cap to JWT_SITES.length also means future site additions or
removals auto-adjust the ceiling.
Two follow-ups to the warm-pool patch:
* Semaphore 1 → 3. Playwright contexts are isolated, the captureToPool
site/port allocation is synchronous (no race), and concurrent user
clicks that all miss the pool no longer serialize behind a single
~5s capture. Peak memory grows from one context to three; each is
short-lived.
* Capture-time validation. After Playwright extracts the JWT, do one
cheap upstream call (/car/info with the public demo VIN) through the
same proxy port before pushing the slot to the pool. DataImpulse
occasionally rotates to IPs the partner widget can load but the
upstream API can't reach, or that get instantly 401/403'd; those
ports used to spend 30s timing out on the first real user click.
Failures rotate to the next site within the existing 4-retry budget.
Adds ~1s to each successful capture; saves up to 30s per dead slot.
The pool was gated to 09:00-19:00 Istanbul so off-hours the slot list was
empty and every first user click paid a 30-120s cold-start tax (Playwright
nav + retries across 4 sites). On the dashboard category page this looked
like an infinite skeleton loader. Pool refresh is cheap (~160 captures/day
per slot) — keep it warm always.
* isBusinessHours() now returns true; scheduleBusinessHours() is a no-op
stub so the onModuleInit call site and businessHoursTimer field stay
valid. All 8 existing gates (initial capture, replacement, refresh
scheduling, refresh skip, scaling) become unconditional.
* getIstanbulTime() dropped — last consumer gone.
* PAGE_TIMEOUT 30s → 10s. Healthy partner sites load in <5s through the
DataImpulse proxy; the longer wait only stretched dead-port retries.
* Drop e-acca.com from JWT_SITES — the current DataImpulse rotating proxy
(74.81.81.81:10000-10999) cannot reach it; page.goto always blocked
until PAGE_TIMEOUT instead of failing fast like the other sites.
Worst-case capture wall-clock: ~120s (4 × 30s) → ~40s (4 × 10s).
First off-hours request, with warm pool: ~120s+ → instant.
The catalog-wide bridge in EmexSourceDbService.fetchCategoryParts was
measured against vehicle_parts on 2026-06-01 and found to return 7-114x
more parts than belong to the requesting vehicle, with 49-98 wrong OEM
codes per 100 served. That directly violates the project rule that the
user must never see a wrong OEM.
Per-catalog noiseRatio sample (catalog-wide / per-vehicle):
RENAULT201910 51x | FFIAT84 45x | VOLVO201410 24x | MB201810 14x
AU1587 8x | BMW202501 70x (+ gid namespace mismatch ETK vs numeric)
GM_C201809 114x | MINI202501 12x | LRE201412 7x | MAZDA2020 54x
GM_OP201809 dump has only 1 wildcard vehicle (unique_key="_") so the
single Crossland X "owns" all 47k Opel parts — same firehose served
to any Opel sub-model in sase prod.
All alternative bridges were proven dead:
SSD eşleştirme - session-bound, 0/91 sase SSDs match dump
scrape_queue_v2.vehicle_ssd - same session SSD format
api_cache replay - table empty (0 rows)
wizard_parameters - table empty (0 rows)
VIN direct - no VIN column in dump
The only viable per-vehicle bridge is vehicles.unique_key reconstruction
from raw_data.parsedOptions, but sase currently stores the required 4
wizard fields on just 5/103 emex vehicles (all Renault). That work is
follow-up; this patch only stops the bleeding.
Change:
- Add EMEX_SOURCE_DB_ALLOWED_CATALOGS env (comma-separated, default "")
- EmexSourceDbService.fetchCategoryParts returns null unless catalogCode
is in the allowlist. Empty allowlist = service is effectively off for
parts, full fallthrough to live emex.
- Connection pool stays alive so the follow-up per-vehicle bridge /
schema-only path can use it without flipping env.
- Boot logs warn loudly when connected with an empty allowlist.
Prod was never affected — CATALOG_SOURCE_DB_ENABLED was unset there. This
fixes dev branch behaviour (default-on since commit 3a3a7d3) and keeps
prod safe by default once main is promoted.
Files:
- packages/config/src/index.ts env schema + audit notes
- apps/api/src/config/configuration.ts parse allowlist into string[]
- apps/api/src/integrations/catalog-source-db/emex-source-db.service.ts
allowlist field, init logging, fetchCategoryParts gate, class doc
- docker-compose.coolify.yml env injection for api + worker
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The browser Sentry SDK initialised fine (DSN reached the bundle,
__SENTRY__ carrier registered) but envelope POSTs were silently
blocked by the existing CSP — `connect-src` didn't list any Sentry
host. Playwright verification on dev.sase.tr confirmed zero requests
to *.sentry.io even after a deliberate uncaught error.
Adds https://*.ingest.de.sentry.io (otolog org lives in the EU/de
region; this matches both the python and sase-web project DSNs).
Verified 2026-06-01 against dev's 103 unique pcat carIds: the current pcat
dump's deep-scrape (7.978 cars with real parts data via schema_parts or
part_groups+part_group_items) targets a US/JDM-market subset — Toyota 2112,
Nissan 1508, Audi 1311, Chevy 1050, Hyundai 745. **None** of sase's TR-market
vehicles intersect that rich subset:
- 18/103 sase carIds are in dump.cars at all (registry only)
- 0/103 yield parts via Bridge A (schema_images → schema_parts)
- 0/103 yield parts via Bridge B (part_groups → part_group_items)
Even the cars that match by exact carId (Fiat Doblo 368 schemas, Renault
Megane, Bravo 456 schemas) have only diagram metadata — no parts annotation.
The dump scraper finished tier-1 (catalog/model/car listing) and tier-2
(schema diagrams) for these, but stopped before tier-3 (parts annotation).
Under the strict "always correct OEM" constraint there is no safe pcat lookup
today. Disable it. The container stays up for future use cases (OEM cross-
reference search, alt-part matching) and so we can flip the env back without
a code change if a richer dump arrives.
EMEX stays on (its catalog-allowlist is the next step).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Easier to see [source-db hit pcat/emex] lines while dev verifies coverage.
Can be downgraded back to debug once we've measured prod hit rates.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Initial design routed emex lookups through vehicles.rawData.ssd → dump vehicles
→ vehicle_parts. Smoke test against prod ssd values: 0 / 10 matched. EMEX
regenerates the SSD on every decode session, so sase's stored SSD never
matches the SSD the dump scraper recorded for the same physical vehicle.
Pivot to a catalog-wide bridge that actually works:
catalogs.code ↔ vehicles.rawData.catalogCode (e.g. "RENAULT201910")
part_groups.group_id ↔ categories.externalId (e.g. "11754")
→ parts via vehicle_parts.group_id (dump's parts.group_id is 100% NULL)
Verified coverage on prod's 8287 unique (catalogCode, gid) pairs: 25/26
catalog codes resolve, 7178 pairs hit a part_group (87%), 5919 of those
return actual parts via vehicle_parts (~71% net). Tradeoff: returns all
parts in the (catalog, group) across every variant in the catalog, so the
result is slightly noisier than the live per-vehicle scrape. Acceptable —
parts overlap heavily and the upstream-call savings outweigh the noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an optional local-dump lookup layer in front of the live PartsCatalogs and
EMEX scrapes. When enabled, getCategoryWithPartsInner queries a Postgres
(pcat) or MariaDB (emex) dump for the requested schema/group's parts and
hotspots; on miss it falls through to the existing upstream call unchanged.
Hits avoid the live API, its cooldown, and its rate-limits — direct DB latency.
- New CatalogSourceDbModule with PcatSourceDbService + EmexSourceDbService
(raw SQL, no Drizzle schema modeling — dump shapes are frozen snapshots).
- pcat lookup keys on schema_images.schema_ext_id (the dump's column that
matches sase's pcat groupId; observed ~7% hit rate on prod's 6596 unique
groupIds, of which ~10% have schema_parts → ~3-5% net parts coverage).
Joins schema_parts → parts directly; the dump's part_groups+part_group_items
linkage covers 0 of our hits, so we skip that path entirely.
- emex lookup uses (catalog_id, ssd) → vehicles.id then (vehicle_id, group_id)
→ vehicle_parts → parts + part_images. The ssd is already persisted into
vehicles.rawData.ssd by the existing emex.mapper, no extra capture needed.
Gated behind CATALOG_SOURCE_DB_ENABLED + PCAT_SOURCE_DB_URL / EMEX_SOURCE_DB_URL.
All three default unset, so this commit is a no-op until prod env is configured.
Adds mysql2 dep for the MariaDB client.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-job moveToDelayed for cooldown livelocked the worker: the activity key is
refreshed by every user request, so 60s later the deferred job comes back, key
is still set, defers again. Last 3h on prod logged ~1800 deferrals against 12
real inits and one completion every ~6 min.
Tag RateLimitError with a `cause`. checkCooldown throws "cooldown"; the worker
now calls `this.worker.rateLimit(delayMs)` and throws Worker.RateLimitError() —
the whole queue waits once instead of cycling every job. checkTimeWindow throws
"time-window"; that branch keeps the existing job.moveToDelayed (per-job) so
EMEX (no scrape window) keeps flowing while PL24/pcat jobs sleep till 09:00.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The live-chat SDK, widget iframe, websocket and avatars are served from
destek.sase.tr; the helmet CSP didn't whitelist it, so the browser
blocked sdk.js (script-src violation) and the widget never loaded.
Add destek.sase.tr to script-/img-/media-/connect-/frame-src (+wss for
ActionCable).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Site-wide live-chat widget served from the self-hosted Chatwoot at
destek.sase.tr, with verified user identity and vehicle context.
- apps/web: lib/chatwoot.ts loads the SDK lazily (mirrors the PostHog
init pattern), init in main.tsx, identify logged-in users in __root
via a server-computed HMAC, and attach the viewed vehicle (VIN/brand/
model) as contact custom attributes on the vehicle detail page.
- apps/api: GET /api/chatwoot/identity (AuthGuard-protected) returns
HMAC-SHA256(user.id) so the widget can use verified identity.
- env: VITE_CHATWOOT_BASE_URL + VITE_CHATWOOT_WEBSITE_TOKEN (build-time,
wired through docker-compose.coolify.yml build args + Dockerfile ARG)
and CHATWOOT_HMAC_TOKEN (api runtime). All optional — widget and
endpoint no-op when unset.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
