Volvo decoded as "Volvo {VIN}" with no real model: the generic Ford-shaped
parseP4VehicleResponse reads window.vehicles/<title>/<h1>, but Volvo ships model/year/type in a
<table id="vinInfoTable"> caption/value grid (Model="V60 Cross Country (19-)", Model yili=2021,
Türü="V60 CC II"). 9 vehicles affected (3 dev + 6 prod).
- Add a per-brand vehicle-info hook (P4BrandHooks.parseVehicleInfo) to the shared P4 engine
(PL24FordLegacyService); brand values win, generic fills gaps. Backward-compatible: no hook → identical.
- New thin PL24VolvoService supplies parseVolvoVinInfo (vinInfoTable parser); orchestrator routes
LEGACY_VOLVO decode to it. Categories/drill unchanged (shared engine).
Verified live vs 6 prod Volvo VINs: all decode real models (S80/S60/S40/V40/EX40·XC40/V60 CC) +
correct years; drill intact (motor → 7 subgroups). First step of the per-brand split
(shared core + thin brand services).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PSA (Peugeot/Citroën/DS) VIN decode was systemically broken: the catalog
vin-group page returns all families unfiltered, so decode fell back to the first
family/salesType (a manual base variant) — yielding "{Brand} {VIN}" model names,
empty transmission, wrong model year, and manual-only parts trees (automatic
gearbox parts missing). Reported for a 1999 Peugeot 106 automatic shown as a 2029
manual with no automatic parts.
- New self-contained PL24PsaService: consumes PL24's FI flow (vin.action →
hintstoken → FI page → json-vin-main-groups → json-vin-illustrations →
vin-image-board). Reads model/year/transmission from the FI identification
table; builds the VIN-indexed parts tree (correct per actual VIN). Does not
touch Ford/Volvo/Nissan/Opel/Hyundai-Kia/Fiat.
- Orchestrator + categories.service route PSA VIN decode/drill to the new service.
- Cycle-correct extractModelYear in @sase/shared (X→1999, not 2029): resolve the
30-yr VIN year code to the most-recent plausible year (≤ now+1); dedupe 6 copies.
Validated live against 13 already-decoded PSA VINs: 12/13 full trees with real
model/year/transmission; automatics correctly detected (106 BVA, 206 AL4, 3008 BVA8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Second batch from undecoded-vin-rca.md. Every entry verified (authoritative
WMI registry + prod sibling + VDS fingerprint + live PL24 probe) before adding
— a wrong brand label or a dead PL24 route both have real downside.
Q5 — WMI_BRAND_MAP (packages/shared, UX/telemetry only; does NOT change which
catalog decodes). Adds 9: W1V=Mercedes-Benz, VXF=Fiat, YAR=Toyota, NL1=Hyundai,
KPA=SsangYong, PL1=Proton, LSV=Volkswagen, LVV=Chery, ZCF=Iveco. Resolves the
RCA conflicts: VXF=Fiat (not Opel), YAR=Toyota ProAce (not Opel); NL1=Hyundai
(not TOGG — MY char predates TOGG). Deliberately NOT added: WF1/WAA/W0W (1-char
corruptions of WF0/WAU/WVW — mapping them would mislabel typo classes), VE1
(unresolved, deferred).
Q6 — PL24_WMI_SERVICE_MAP (real decode routing). Both target services are
proven live, so these only append WMIs to working routes (no dead-route →
isDecodeable()=true → 25s-budget/circuit-breaker hazard):
- SJN/VSK/MNT → nissan_parts (Sunderland/Spain/Thailand) — regression restore,
dropped in the 757905f refactor; nissan_parts proven (JN1 decodes in prod).
- W1V → mercedesvans_parts (Sprinter/Vito) — retires the RCA "never tried" flag:
mercedesvans_parts decodes WDF44770513186937 live today via the same P5 path.
Tests: shared getBrandFromWmi (adds + corruption-skips return null), pl24 route
map. Full api suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ConfigService.get<number>("EMEX_PROXY_PORT_START") returns the raw env
STRING; the port-pick arithmetic then string-concatenated it
(45 + "10001" = "4510001"), producing an out-of-range port that made
undici's `new URL` throw "Invalid URL" at EmexService construction —
crashing the entire API on boot.
A single-port range (823) happened to concat to a still-parseable "0823",
which masked the bug for months. It surfaced the moment the prod
EMEX_PROXY_PORT range was widened (823 -> 10001-10099) to let Q1's
per-request port rotation work: prod crash-looped until the env was
reverted. Coerce to a validated integer port (1-65535) with default
fallback so a real range is safe.
Regression test: constructing EmexService with string port env over a
real range must not throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-cause analysis live re-decoded all 124 historically-undecoded prod
VINs; 38 already decode now. These 4 fixes target confirmed code bugs
that drop or mask real decodes (see undecoded-vin-rca.md):
Q2 — PL24 circuit breaker now only counts transient transport faults. A
definitive upstream negative (NotFound/BadRequest) no longer trips the
global 30s breaker that was starving PL24 for every subsequent VIN
(the sibling-VIN inconsistency in the report). Live-proven on VR7.
Q3 — previewVin / multi-candidate path no longer returns an empty
success: the pcat/emex candidate branches fill brandName (from catalogId
/ WMI), fixing the 6 "HTTP 200 with null brand+model" cases.
Q1 — EMEX fetch retries transient proxy failures with a FRESH ProxyAgent
per attempt (rotates the DataImpulse port; ~42% blip rate observed),
plus an opt-in direct fallback (EMEX_DIRECT_FALLBACK). HTTP answers are
never retried.
Q4 — VIN resolve cache keys namespaced by DECODE_CHAIN_VERSION and the
negative TTL drops 6h -> 30m, so a decode-chain fix self-heals stale
negatives on deploy instead of masking phantom-undecoded VINs for hours.
The admin cache-buster uses the same key builder.
Tests: 179 passed (+ new Q2/Q3/Q4 specs). typecheck + biome clean.
Deploy note: prod EMEX_PROXY_PORT_START/END are both 823 (single port);
widen to a real range (e.g. 10001-10099) in Coolify so Q1's port
rotation takes full effect.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The worker's own upstream fetches called touchActivity(), setting the
prefetch:activity:<source> cooldown key (TTL 300s) that checkCooldown then
honoured — so after each fetch the worker paused itself for up to ~5 minutes
(re-checking every 60s, ~5 empty cycles per key). At ~1 fetch / 5 min the
3387-job backlog needed ~6 days to drain.
- Wrap each worker job in an AsyncLocalStorage backfill context; touchActivity
skips the cooldown key when invoked from the worker, so the cooldown reflects
only real user requests (worker yields to users, never to itself).
- Cooldown TTL 300s -> 90s (a request 5 min ago isn't "active").
- checkCooldown pauses for the key's actual remaining TTL (one wait) instead of
a fixed 60s re-check loop.
No extra upstream load — only removes the worker's self-imposed idle time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Volvo's vin-image-board.action ships its BOM inline as partno= tc-data-row
rows (no pncHierCode / json-vin-bom-detail), so the Ford VIN-BOM parser
returned 0 parts. Fall back to parsePsaBomParts (partno= rows) when no pnc
rows are found. Also decode HTML entities (Ö " …) in scraped Volvo
subgroup names. Completes the Volvo chain: group1→group2→illustration→parts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bugs in the vin-group.action subgroup scrape: (1) the filter dropped any
href containing "openVinDialog", but the real sub-group links carry
openVinDialog=false (only the VIN-dialog crumb is =true) — so every child was
discarded; (2) the deepest group level lists its illustration leaves as
vin-image-board.action links, which weren't extracted. Now match both deeper
vin-group.action?groupN= and vin-image-board.action anchors, and only drop the
openVinDialog=true crumb. Completes Volvo group1→group2→illustration→parts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PL24 serves a stripped NOT_LOGGED_IN_DEMO page (no groups/parts) when the
service token is stale. decodeVinForService retries on this, but drill paths
(fetchSubGroupsByPath/fetchPartsByPath) reach upstream only via fetchP4Page,
which didn't — so Volvo subgroup drilling parsed empty demo pages. Retry once
with fresh auth on a demo page. Adds a Volvo-drill diagnostic log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Volvo vin-group.action categories store hrefs relative to the catalog dir
(e.g. "vin-group.action?group1=…"). fetchP4Page did `${baseUrl}${url}`,
collapsing to "partslink24.comvin-group.action" → ENOTFOUND. Prefix the
service basePath when the path is relative. Fixes Volvo subgroup drilling
for existing (relative) stored linkPaths without a re-decode.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Volvo's VIN catalog is a 3-level vin-group.action?group1=…[&group2=…] HTML
tree; PL24's json-vin-*-group.action JSON endpoints now 404. Add HTML
subgroup extraction (keep links one group-level deeper than the current
path) in ford-legacy fetchSubGroupsByPath, and route vin-group.action?group1=
nodes through getChildren (drill-first, fall back to parts) in
getCategoryWithParts. Recovers Volvo group1→group2 navigation. Leaf parts
handled separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Volvo (VIN-indexed legacy catalog) ships its real top groups as
vin-group.action?group1=… in the vin-group HTML. PL24's
json-vin-main-group.action endpoint now 404s, so decode falls back to the
HTML scrape — but parseP4NavigationCategories AND the seed-time NAV_CRUMB
filter both blanket-exclude vin-group.action, dropping every real Volvo
group → 0 categories. Exclude vin-group.action as a crumb only when it
lacks group1=. Confirmed upstream: 7 real groups (Frenler, Elektrik
sistemi, …) present in the HTML for YV1AS7050A1118639.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
isP4LegacyPath matched any ".action" path, shadowing the PSA
json-illustrations.action / image-board.action dispatch in
fetchSubGroupsByPath and fetchPartsByPath. Every PSA (Citroen/Peugeot/DS)
drill below main-group level fell through to the Ford/Fiat legacy parser,
which cannot parse PSA JSON, so it returned empty. PSA vehicles decoded
since the catalog module landed (72c0de6) showed categories but 0 parts
(41/43 affected). Exclude /psa/ from isP4LegacyPath so these paths reach
fetchPsaIllustrations / fetchPsaParts. Heals existing vehicles on demand;
no re-decode needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The post-goto token poll ran a blind 20s wait regardless of whether
page.goto succeeded or threw. On a healthy goto the widget API call
fires within ~1-2s; on a failed goto the request either already went
through (rare) or never will (common). The 20s cap was the dominant
cost on failed-site attempts — verified tonight as a 28s "No token
after ..." log on auto-komplekt after page.goto ERR_TIMED_OUT.
* CAPTURE_POLL_AFTER_OK = 10 (5s) — token usually arrives in <2s
* CAPTURE_POLL_AFTER_FAIL = 4 (2s) — brief grace then bail
Per-attempt worst case on a dead site: 10s goto + 2s grace = 12s
(was 10s + 20s = 30s). On a healthy site, well-known capture times
(3-5s) stay comfortably inside the 5s post-goto cap.
A captured slot now lives in Redis under `pcat:jwt:slot` for the JWT's
remaining lifetime (minus a 60s safety buffer). On module init we try
Redis before launching Playwright — if a fresh slot is there we adopt it
and schedule its refresh, skipping the ~5s capture entirely. After every
successful capture+validation we publish to Redis so the next restart (or
any sibling pod) can inherit. invalidateSession deletes the Redis copy
because a 401/403 means the cached IP-binding is dead.
Token is still IP-bound to its proxyPort. If a hydrating container reads
the slot but the proxy has rotated away from the captured IP, the next
upstream call 401s and the existing invalidateSession fallback re-captures
locally — so worst case = today's cold-capture behavior, never worse.
Note: dev and prod use separate Redis instances. This patch reaches PL24
parity (same-env redeploy hydration); a true dev↔prod shared cache would
need either an external Redis or an internal-token bridge.
Concurrent captures are bound by how many distinct partner sites we can
drive in parallel — each capture needs its own site so they don't collide
on the synchronous siteIndex round-robin. Ports are effectively unlimited
(10000-10999) and Playwright contexts are isolated, so the previous
arbitrary cap of 3 left headroom on the table when the pool is cold and
N>3 user clicks race in at once.
Tying the cap to JWT_SITES.length also means future site additions or
removals auto-adjust the ceiling.
Two follow-ups to the warm-pool patch:
* Semaphore 1 → 3. Playwright contexts are isolated, the captureToPool
site/port allocation is synchronous (no race), and concurrent user
clicks that all miss the pool no longer serialize behind a single
~5s capture. Peak memory grows from one context to three; each is
short-lived.
* Capture-time validation. After Playwright extracts the JWT, do one
cheap upstream call (/car/info with the public demo VIN) through the
same proxy port before pushing the slot to the pool. DataImpulse
occasionally rotates to IPs the partner widget can load but the
upstream API can't reach, or that get instantly 401/403'd; those
ports used to spend 30s timing out on the first real user click.
Failures rotate to the next site within the existing 4-retry budget.
Adds ~1s to each successful capture; saves up to 30s per dead slot.
The pool was gated to 09:00-19:00 Istanbul so off-hours the slot list was
empty and every first user click paid a 30-120s cold-start tax (Playwright
nav + retries across 4 sites). On the dashboard category page this looked
like an infinite skeleton loader. Pool refresh is cheap (~160 captures/day
per slot) — keep it warm always.
* isBusinessHours() now returns true; scheduleBusinessHours() is a no-op
stub so the onModuleInit call site and businessHoursTimer field stay
valid. All 8 existing gates (initial capture, replacement, refresh
scheduling, refresh skip, scaling) become unconditional.
* getIstanbulTime() dropped — last consumer gone.
* PAGE_TIMEOUT 30s → 10s. Healthy partner sites load in <5s through the
DataImpulse proxy; the longer wait only stretched dead-port retries.
* Drop e-acca.com from JWT_SITES — the current DataImpulse rotating proxy
(74.81.81.81:10000-10999) cannot reach it; page.goto always blocked
until PAGE_TIMEOUT instead of failing fast like the other sites.
Worst-case capture wall-clock: ~120s (4 × 30s) → ~40s (4 × 10s).
First off-hours request, with warm pool: ~120s+ → instant.
The catalog-wide bridge in EmexSourceDbService.fetchCategoryParts was
measured against vehicle_parts on 2026-06-01 and found to return 7-114x
more parts than belong to the requesting vehicle, with 49-98 wrong OEM
codes per 100 served. That directly violates the project rule that the
user must never see a wrong OEM.
Per-catalog noiseRatio sample (catalog-wide / per-vehicle):
RENAULT201910 51x | FFIAT84 45x | VOLVO201410 24x | MB201810 14x
AU1587 8x | BMW202501 70x (+ gid namespace mismatch ETK vs numeric)
GM_C201809 114x | MINI202501 12x | LRE201412 7x | MAZDA2020 54x
GM_OP201809 dump has only 1 wildcard vehicle (unique_key="_") so the
single Crossland X "owns" all 47k Opel parts — same firehose served
to any Opel sub-model in sase prod.
All alternative bridges were proven dead:
SSD eşleştirme - session-bound, 0/91 sase SSDs match dump
scrape_queue_v2.vehicle_ssd - same session SSD format
api_cache replay - table empty (0 rows)
wizard_parameters - table empty (0 rows)
VIN direct - no VIN column in dump
The only viable per-vehicle bridge is vehicles.unique_key reconstruction
from raw_data.parsedOptions, but sase currently stores the required 4
wizard fields on just 5/103 emex vehicles (all Renault). That work is
follow-up; this patch only stops the bleeding.
Change:
- Add EMEX_SOURCE_DB_ALLOWED_CATALOGS env (comma-separated, default "")
- EmexSourceDbService.fetchCategoryParts returns null unless catalogCode
is in the allowlist. Empty allowlist = service is effectively off for
parts, full fallthrough to live emex.
- Connection pool stays alive so the follow-up per-vehicle bridge /
schema-only path can use it without flipping env.
- Boot logs warn loudly when connected with an empty allowlist.
Prod was never affected — CATALOG_SOURCE_DB_ENABLED was unset there. This
fixes dev branch behaviour (default-on since commit 3a3a7d3) and keeps
prod safe by default once main is promoted.
Files:
- packages/config/src/index.ts env schema + audit notes
- apps/api/src/config/configuration.ts parse allowlist into string[]
- apps/api/src/integrations/catalog-source-db/emex-source-db.service.ts
allowlist field, init logging, fetchCategoryParts gate, class doc
- docker-compose.coolify.yml env injection for api + worker
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Verified 2026-06-01 against dev's 103 unique pcat carIds: the current pcat
dump's deep-scrape (7.978 cars with real parts data via schema_parts or
part_groups+part_group_items) targets a US/JDM-market subset — Toyota 2112,
Nissan 1508, Audi 1311, Chevy 1050, Hyundai 745. **None** of sase's TR-market
vehicles intersect that rich subset:
- 18/103 sase carIds are in dump.cars at all (registry only)
- 0/103 yield parts via Bridge A (schema_images → schema_parts)
- 0/103 yield parts via Bridge B (part_groups → part_group_items)
Even the cars that match by exact carId (Fiat Doblo 368 schemas, Renault
Megane, Bravo 456 schemas) have only diagram metadata — no parts annotation.
The dump scraper finished tier-1 (catalog/model/car listing) and tier-2
(schema diagrams) for these, but stopped before tier-3 (parts annotation).
Under the strict "always correct OEM" constraint there is no safe pcat lookup
today. Disable it. The container stays up for future use cases (OEM cross-
reference search, alt-part matching) and so we can flip the env back without
a code change if a richer dump arrives.
EMEX stays on (its catalog-allowlist is the next step).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Initial design routed emex lookups through vehicles.rawData.ssd → dump vehicles
→ vehicle_parts. Smoke test against prod ssd values: 0 / 10 matched. EMEX
regenerates the SSD on every decode session, so sase's stored SSD never
matches the SSD the dump scraper recorded for the same physical vehicle.
Pivot to a catalog-wide bridge that actually works:
catalogs.code ↔ vehicles.rawData.catalogCode (e.g. "RENAULT201910")
part_groups.group_id ↔ categories.externalId (e.g. "11754")
→ parts via vehicle_parts.group_id (dump's parts.group_id is 100% NULL)
Verified coverage on prod's 8287 unique (catalogCode, gid) pairs: 25/26
catalog codes resolve, 7178 pairs hit a part_group (87%), 5919 of those
return actual parts via vehicle_parts (~71% net). Tradeoff: returns all
parts in the (catalog, group) across every variant in the catalog, so the
result is slightly noisier than the live per-vehicle scrape. Acceptable —
parts overlap heavily and the upstream-call savings outweigh the noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an optional local-dump lookup layer in front of the live PartsCatalogs and
EMEX scrapes. When enabled, getCategoryWithPartsInner queries a Postgres
(pcat) or MariaDB (emex) dump for the requested schema/group's parts and
hotspots; on miss it falls through to the existing upstream call unchanged.
Hits avoid the live API, its cooldown, and its rate-limits — direct DB latency.
- New CatalogSourceDbModule with PcatSourceDbService + EmexSourceDbService
(raw SQL, no Drizzle schema modeling — dump shapes are frozen snapshots).
- pcat lookup keys on schema_images.schema_ext_id (the dump's column that
matches sase's pcat groupId; observed ~7% hit rate on prod's 6596 unique
groupIds, of which ~10% have schema_parts → ~3-5% net parts coverage).
Joins schema_parts → parts directly; the dump's part_groups+part_group_items
linkage covers 0 of our hits, so we skip that path entirely.
- emex lookup uses (catalog_id, ssd) → vehicles.id then (vehicle_id, group_id)
→ vehicle_parts → parts + part_images. The ssd is already persisted into
vehicles.rawData.ssd by the existing emex.mapper, no extra capture needed.
Gated behind CATALOG_SOURCE_DB_ENABLED + PCAT_SOURCE_DB_URL / EMEX_SOURCE_DB_URL.
All three default unset, so this commit is a no-op until prod env is configured.
Adds mysql2 dep for the MariaDB client.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- WMI_BRAND_MAP + getBrandFromWmi packages/shared'e taşındı (tek kaynak);
corgi.service artık buradan import ediyor (davranış aynı, testler geçiyor)
- VinBrandIcon: VIN'in WMI'ı bilinen markaya denk gelince büyüteç yerine
marka logosu pop animasyonuyla görünür (landing + search VIN textbox)
- WMI haritasına prod DB'de decode edilmiş VIN'lerden 23 eksik WMI eklendi
(Türkiye fabrikaları NM4/NMT/NLA/NLH/NMB dahil; tüm DB WMI'ları artık tanınıyor)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The DataImpulse proxy has dead ports that stall ~10s on connect, and undici's
ProxyAgent connect.timeout does NOT bound the proxy connection (verified: runtime
still 10s despite the Phase-2 setting). With the 30s request timeout, 2-3 dead
ports blow the 25s decode budget → false misses + 12-32s p95 (prod). /car/info
answers in <1s on a healthy proxy, so bound it at the fetch level instead: a 6s
per-call AbortSignal + 4 retries (both env-tunable: PCAT_DECODE_TIMEOUT_MS /
PCAT_DECODE_MAX_RETRIES) make a stuck port abort fast and rotate to a live one
within budget. Helps every PCAT decode, not just Renault.
Band-aid for proxy flakiness; proper fix is proxy port health-tracking (follow-up).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PL24 has suspended Renault VIN identification ("...belirsiz bir süre için mevcut
olmayacağını üzülerek bildiririz") — renault_parts authorizes but every decode
throws that message. In the race that's a wasted ~1s call AND, worse, it counts
as a PL24 failure: 3 strikes opens the PL24 circuit breaker, which then skips
PL24 for ALL brands (Opel/Ford/PSA…) for 30s. Prod data: VF1 (Renault) was the
single biggest decode-failure WMI, much of it repeat hits on a poisoned cache.
Comment out the renault_parts WMIs (VF1/VF6/VNE) so PL24 isn't tried for Renault;
PCAT + EMEX cover it. Re-enable when PL24 restores Renault VIN decode.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PL24 account JWTs were in-memory only, so every (re)deploy forced a fresh login
on the next decode, and api + worker each logged in independently — fighting over
PL24's one-session-per-account (squeezeOut). Persist account tokens to Redis
(keyed by account, TTL = JWT exp) and hydrate from Redis before logging in.
Best-effort: any Redis error falls through to a normal login, so auth never
depends on Redis. Service tokens stay in-memory (warmed by the boot pre-warm).
Complements the OnModuleInit pre-warm — now even the base login survives restarts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PL24 auth tokens (account JWT + per-service tokens) are cached in-memory, so a
(re)deploy clears them and the first VIN decode of each service-family pays the
~10s login + service-authorize handshake on the request path. Measured on dev: a
cold opel_parts decode took 12s vs ~2s once the auth was warm.
Add OnModuleInit to PL24AuthService that warms, in the background (fire-and-forget,
never blocking boot), both base account logins (the shared dominant cost) plus the
common legacy + top Turkish-market service tokens. allSettled throughout so a
slow/down PL24 degrades gracefully; gated on credential presence; PL24_PREWARM=false
disables. The pre-decode (search page) already hides this from users mid-session —
this closes the one remaining gap: the first dealer right after a deploy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prod decode failures clustered on French/Stellantis VINs that no source even
tried: the post-merger WMIs VR3/VR7 (Peugeot), VR1 (Citroën/DS) and W0V (Opel)
were missing from every map, so getServiceName/isSupported returned null and the
VIN was rejected as "Marka desteklenmiyor" before any scrape ran.
Add them to all three maps, matching the existing VF3/VF7/W0L routing:
- Corgi WMI_DATABASE (display brand)
- PL24_WMI_SERVICE_MAP (peugeot_parts / citroen_parts / opel_parts)
- EMEX CATALOG_MAP (PSA -> PEUGEOT00, Opel -> GM_OP201809)
Lets EMEX scrape and PL24 fall back for these mass-market TR brands instead of
hard-failing at the routing step. Phase 4 of 4 on decode reliability.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PartsCatalogs ProxyAgent had no connect timeout, so a dead DataImpulse proxy
port stalled for undici's 10s default before the retry rotated to a fresh port
(prod: "ConnectTimeoutError ... 74.81.81.81:10628, timeout: 10000ms"). Three such
stalls exceed the caller's 25s decode budget, turning a transient bad port into a
hard decode failure.
Set an explicit 6s proxy connect timeout so retries rotate to a live port within
budget, and make the request timeout / retry count / connect timeout env-tunable
(PCAT_REQUEST_TIMEOUT_MS, PCAT_MAX_RETRIES, PCAT_PROXY_CONNECT_TIMEOUT_MS) with the
current values as defaults. Phase 2 of 4 on decode reliability.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A decode that failed from a transient proxy/network blip (PartsCatalogs
DataImpulse ConnectTimeout, EMEX timeout, PL24 transport error) was written to
the 6h negative cache identically to a genuine "brand not supported" miss. A
user who hit a blip then got "Şase tanınamadı" for 6h with no way to retry out —
and prod analytics showed this hitting SUPPORTED brands (Fiat/Toyota/Hyundai
decode fine once the proxy responds).
Thread a `transient` outcome flag through resolveVin -> doResolveVin:
- PartsCatalogs.decodeVin sets it when it swallows a transport error (vs a clean
"no cars" miss), via a new optional outcome param.
- EMEX surfaces it via its existing {type:"error"} result.
- PL24 sets it on transport/timeout errors in the fallback catch.
resolveVin then skips the negative cache when transient=true; genuine misses
still cache for 6h. Phase 1 of 4 on decode reliability.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The parts-catalogs HTTP client only retried request timeouts and 401/403;
undici network errors ("TypeError: fetch failed" — a dropped/reset
DataImpulse proxy connection) were thrown on the first attempt. During the
cold-JWT window (outside 09:00-19:00 Istanbul) the proxy drops frequently,
so a single click would surface an empty/slow category that returns full
parts on a manual retry — and could even mark a populated parent group
unavailable when child discovery hit the blip.
Retry transient transport errors (timeout + undici network failures) with a
short backoff, re-acquiring a session each attempt. Definitive HTTP
responses (re-thrown as `Error: HTTP <code> …`, e.g. 400 "list of parts is
empty") are still thrown immediately — they are real answers, not blips.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PCAT was missing parity with EMEX in two places:
- Part `description` (notice) was stored raw English alongside translated `name`.
Now batched into translateMany so users see Turkish notices.
- Vehicle body/engine/transmission attrs from VIN decode were raw upstream
values. Wire up the existing emex.mapper dictionaries on the PCAT
single-car, PCAT resolveById, and EMEX single-vehicle result paths.
- emex.mapper translateToTurkish now falls back to the original term on
dictionary miss instead of null — upstream values are heterogeneous
(engine codes, multi-word descriptors); losing them was worse than
leaving them untranslated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
MAN truck (p5man) main-vin and sub-vin endpoints return human-readable
names in values.mainGroupDescription and values.groupDescription. Our
parsers only checked the standard caption/description fields used by
VW/Renault/etc., so MAN categories were stored with raw numeric codes
("0", "1", ..., layoutIds like "2884606") as names.
After this change, the user sees proper Turkish names like
"MOTOR, SOĞUTMA SİSTEMLERİ" and "ANTEN". Existing brands fall through
the same field chain as before — captionMatch[2] takes precedence,
keeping their behavior unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PCAT auth pool warm-up always started with e-acca.com, which doesn't
load through the new DataImpulse rotating proxy — page.goto sits on
the 30s navigation timeout and only then falls through to the next
candidate. The remaining sites (alkatalog, auto-komplekt, autotrade,
…) reach the upstream widget in 3-10s through the same proxy, so an
on-demand acquireSession() that lands during a cold start spends 30s
blocked on the dead first hop before any retry can succeed.
Move e-acca to the end of the list. New cold-pool capture finishes
in ~4s on the first reachable site (alkatalog) instead of 30s+9s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Probe (scripts/dev) confirmed that everything PartsPanel reads from an
emexdwc.ae leaf — parts table, hotspot coordinates, and the schema image
URL — is fully server-rendered in Unit.aspx HTML. Parts arrive as
`<tr name>` rows with `td[name=c_oem|c_pnc|c_name]`; hotspots are
`<div class="dragger g_highlight" name=N style="margin-top:Ypx; margin-left:Xpx; ...">`
with the coords already in image-natural pixel space; the image URL is
in `<img class="dragger" src=...laximo...>` and its native dims can be
read from the first 24 bytes of the GIF/PNG via a Range GET.
Behaviour:
1. fetchCategoryParts now tries fetchCategoryPartsViaHttp first — two
sequential GETs (QuickDetails → Unit) + a Range GET for image dims.
2. If the HTML yields ≥1 part, we return it.
3. If the HTML returns no Unit.aspx anchor, or 0 parts, we fall back to
the existing Playwright scraper (same code path as before).
The plain-HTTP path skips the ~1-2s browser launch, sidesteps the
3-page semaphore in EmexBrowserService (concurrency cap was throttling
prefetch fan-out), and uses no chromium memory. Measured on dev with 5
fresh-ssd Renault Espace IV leaves: 4.9-5.5s wall per leaf (vs 6-7s on
the Tier 1 browser path, vs 12-14s pre-Tier-1). The 6th sample
(stale-ssd Fren Kaliyeri) failed both paths identically — confirms the
plain-HTTP path doesn't introduce new failure modes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EMEX-decoded vehicles surfaced empty category pages on prod because two
runtime prerequisites were absent from the production image:
1. `scripts/emex-vin-scraper.js` was never copied — the build stage's
`COPY . .` brings it in but the production stage only cherry-picks
`apps/api/dist`, `drizzle`, and `start.sh`. Every EMEX leaf hit
therefore failed with "Scraper file not found at: /app/scripts/…"
and the category page rendered "Bu kategori icin parca bulunamadi."
2. `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH=/usr/bin/chromium-browser` was
exported in the Dockerfile but never reached `chromium.launch()` —
that env var is consumed by `playwright install`, not at runtime.
Playwright fell back to its bundled headless-shell cache path
(`/root/.cache/ms-playwright/chromium_headless_shell-*/…`) which
does not exist on the alpine image, so even with the scraper file
present the browser pool init would have kept failing.
Fix:
- Dockerfile: `COPY --from=build /app/scripts ./scripts`.
- `emex.browser.ts` + `parts-catalogs-auth.service.ts`: read
`process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` and pass it as
`executablePath` to `chromium.launch()` when set.
Verified on prod container: `ls /app/scripts` → missing pre-fix; the
binary at `/usr/bin/chromium-browser` exists, so the env-var hand-off
will resolve cleanly once the new image lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fetchFordVinBomParts was forwarding the relative HTML jsonUrl with
isFullUrl=true, which made undici try to parse \`/ford/fordp_parts/…\`
as an absolute URL and throw "Failed to parse URL" for every PNC row.
The Hyundai BOM expander next door calls fetchP4Page with the default
isFullUrl=false (relative path) — match that.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
VIN-decoded Ford P/T and Volvo legacy vehicles used to land on an empty
category page: the .action HTML the decoder scraped only carried header
breadcrumbs (Portal / Model seçimi / VIN). The real catalog hangs off
three JSON endpoints that the partslink24 UI calls in the background
once a VIN session is established. None of them need mode/upds/JSESSIONID
beyond the standard PL24TOKEN cookie. Plumb the whole chain so a user
who decoded a VIN sees real Turkish part categories and OEM part
numbers in the user-vehicle flow:
json-vin-main-group.action → real top-level groups (8 for Mondeo)
json-vin-sub-group.action → 58 leaf subgroups (filters subheaders)
vin-image-board.action → BOM table with pncHierCode + jsonUrl
json-vin-bom-detail.action → final OEM partno entries (per variant)
Schema image fetch reuses the existing image-ticket extractor since the
ticket URL lives in the same jsIlluData payload as Hyundai/Opel/Volvo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
VIN-group HTML for Ford/Volvo legacy services exposes the header
breadcrumb links (Portal / Model seçimi / current VIN) via the same
.action pattern the parser relied on, so they were being persisted as
"part categories" — users saw three useless rows instead of real groups.
The PSA flow already documented this trap and bypassed rawData; extend
the same defense to the rest of P4 legacy (parser-side + categories
insert fallback).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnostic served its purpose: confirmed that PL24 bomOverviewlist
URLs (e.g. illustrationId=4336125, "Alternatöre genel bakış" with
PR-code filters) return only a records array with no images field —
upstream genuinely has no schema illustration for these overview
tables. Frontend UX for this case is handled separately.
Production p99 was 18min for failed decodes and 15min for PL24 successes;
same VIN could trigger N parallel 10+ min decodes. Three layered fixes:
- In-flight dedup via Redis SETNX (vin🔒*); concurrent same-VIN requests
poll the resolve cache instead of re-firing the upstream chain.
- 24h positive cache (vin:resolve:*) and 6h negative cache
(vin:resolve:neg:*); previously 5min positive / no negative.
- 25s hard abort budget via AbortController; PCAT gets the signal natively
(AbortSignal.any), PL24/EMEX wrapped with raceWithSignal at the boundary.
Aborted decodes don't poison the negative cache.
- PCAT/EMEX real race: first definitive single-result wins; the slower
source is skipped (previously PCAT was always awaited first).
- PL24 circuit breaker: 3 consecutive failures opens a 30s cooldown
(pl24:cb:cooldown_until); successes reset the counter.
- Stage-level timings in query_logs.timings (jsonb): pcat/emex/pl24/
lock_wait/cache_hit/aborted. Failed source now logged as "none" or
"aborted" instead of misleading "corgi".
Verified locally with 3 parallel decodes of a fresh VIN: 1 real decode
(3.59s), 2 lock-waits (3.53s) sharing the result, 4th request 23ms cache
hit. Previously this would have been 3 separate 10+ min PL24 decodes.
Migration 0002 adds query_logs.timings jsonb (NULL default). Must be
applied manually before deploy (deploy.sh does not run db:push).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The biome safe-fix sweep in 3184e4c rewrote `import { X }` to
`import type { X }` for several NestJS providers (ConfigService,
Reflector, EmailService, Database, etc.). Type-only imports are
erased at compile time, but Nest's reflect-metadata DI reads
constructor parameter types at runtime, so every affected provider
threw "Nest can't resolve dependencies (?)" and the API crash-
looped under PM2 (377+ restarts).
Convert all `import type {` to `import {` in apps/api/src so the
type tokens survive into the emitted JS for runtime DI.
Note: biome's useImportType rule does not understand the
reflect-metadata pattern. Worth disabling for apps/api/ in a
follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Run 'biome check --fix' on apps/api/src and apps/web/src to clear
the safe-fixable lint backlog (151 files: parseInt → Number.parseInt,
isNaN → Number.isNaN, organize imports, etc.). 769 errors remain
that require manual changes (mostly noExplicitAny).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- CATALOG_MAP'e ~40 yeni WMI eklendi (Mitsubishi, Nissan, Volvo, vb.)
- brandFromCatalogCode listesi 55 DB kataloğuna genişletildi
- brand="UNKNOWN" olsa bile catalogCode varsa EMEX sonucu kabul edilir
- emex-catalog: linkPath'li kategorilerde parçalar DB'den yüklenir (scraper bug fix)
- VIN cache: emex_vehicle_vins tablosu ile tekrar VIN'lerde emexdwc.ae atlanır
- getCategoryTree Step 0: VIN cache lookup (~10ms)
- Match sonrası otomatik VIN cache doldurma
- Proaktif cache: ilk decode sonrası matchByName + saveVinCache
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
