Prod decode was dominated by the DataImpulse proxy: ~50% of ports dead at any
moment, 57% of captured tokens discarded by validation, the dead-port slot
never evicted (invalidateSession only fired on 401/403, never on transport
timeout), and a cold pool blocked the user inline on a multi-second capture.
pcat sub-timing was p50 8.3s / p90 56s, blowing the 25s decode budget.
Verified on prod: the TWS- token is NOT request-time IP-bound (0 auth 401/403
across thousands of calls; a token captured on one IP replays 200 through
arbitrary fresh residential IPs). So the proxy can rotate freely.
- Transport swap to Floxy residential (PCAT_PROXY_PROVIDER=floxy|dataimpulse|none).
Call/validate leg = a fresh exit IP per request (max IP diversity → min per-IP
ban); capture leg = a sticky session per Playwright attempt. DataImpulse +
direct kept as env fallbacks. Token decoupled from the slot (proxyPort removed).
- Prune 3 zero-yield JWT sites (knkauto/autodo.kz/flynestauto — 0 captures, ~80%
of attempts); add PCAT_JWT_SITES env override.
- Lower catalog-scrape timeout 30s→10s (env-tunable) so a slow IP aborts within
budget and the retry rotates to a fresh IP.
- Cold-pool fast-fail on the decode race: warm in background, let EMEX/PL24
answer instead of blocking inline; treated as a transient miss (no neg-cache).
- Evict a token after N consecutive transport failures on rotating IPs (the
dead-slot bug); reset on any 2xx.
- PCAT circuit breaker mirroring PL24 (transient-only; cold-skip neutral; skip
marks outcome.transient so a pcat-only VIN isn't negative-cached 30m).
Typecheck + Biome + unit tests (categories/vehicles) green. Default flips pcat
to Floxy on next deploy; instant rollback via PCAT_PROXY_PROVIDER.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Customer entered VXKUPHNKSP4032040 (2023 Corsa F) and got a bare
'Marka desteklenmiyor' — VXK was missing from all three WMI maps, so
identifyBasic could not even name the brand. PL24's opel_parts catalog
ends at CORSA-E (2015-2019, verified via the model-picker page), so the
catalog itself stays a gap, but the user now gets the honest
'Opel 2023 olarak tanındı, katalog henüz mevcut değil' message and the
miss is logged for the coverage backlog.
- WMI_BRAND_MAP + PL24_WMI_SERVICE_MAP + EMEX CATALOG_MAP: VXK -> Opel
- DECODE_CHAIN_VERSION 2 -> 3 (brand-mapping change; invalidates the
stale negative cache for the customer's VIN)
- vin-cache-keys.spec: lock-key assertion checked the bare version char,
which now appears inside the test VIN — assert the ✌️ segment instead
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the EAN line from the matched-article cards and stop fetching/
shipping eans in /p/oem (its only consumer was that line). Snapshot
still holds the data; the endpoint just no longer queries it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
OEM detail "slow load" was two things, not the query (DB 135ms / API
13-120ms):
1. The parts-panel link opened a NEW TAB → full SPA cold boot every
click. Switch to in-app client navigation on plain click (real href
kept, so ctrl/cmd/middle-click still opens a new tab).
2. The /p/oem response shipped each article's oeNumbers + compatible
lists (up to 200 each × 60 articles) that the UI never renders — 96%
of a 370 KB payload. Ship lean articles; aggregates already carry
the cross-refs. 60-article code: 370 KB → ~12 KB.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every OEM code was a link → unmatched codes (the majority — fasteners,
clips, body parts TecDoc doesn't carry) opened an empty "no
equivalents" page. Now the schema page batch-checks its codes against
the snapshot (POST /p/matched, one indexed query) and links ONLY the
matched ones; unmatched codes render as plain text. Copy stays for all.
No more dead-end empty pages from the parts panel.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finishes 3b14431 (which only captured the file renames): apply the
identifier/endpoint/env/UI changes so the code matches the new paths
— PModule/PController/PSourceDbService, @Controller("p"), /p/oem,
config key `p`, P_DB_ENABLED/P_DB_URL, "P kataloğundan…" copy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drop the third-party brand name from the product surface and code.
The OEM detail feature is now "P" (short for parts) everywhere:
endpoint /p/oem, PModule/PController/PSourceDbService, config key
`p`, env P_DB_ENABLED/P_DB_URL, UI copy ("P kataloğundan…"), and the
unused tab label. Physical snapshot DB stays `td` (server-only).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The snapshot stores scrape-local '/_debug/...' image paths that 404
off-host. Filter the OEM lookup to http(s) image URLs and render the
thumbnail only when one exists, so the detail page shows clean
code-focused cards instead of a grid of broken-image placeholders.
Real CDN URLs will surface automatically once present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve a catalog OEM code to its TecDoc equivalents on a new
/dashboard/oem/$code page: the aftermarket parts that carry it
(brand + article number + image + EAN), buyable supplier
substitutes, and OE cross-references (same part under other makes).
- API: TecdocModule (read-only postgres-js client to the imported
`td` snapshot), GET /tecdoc/oem?code=. Normalisation-based match
(TecDoc stores `1J0 973 702`, catalog gives `1J0973702`); exact
match recovers ~1/10 vs normalised ~5/10 on real codes. Self-
disables without TECDOC_DB_* env → { matched: false }.
- Web: OEM code in the parts panel is now a link (new tab) to the
detail page; "N/A" stays plain text.
- Mirrors CatalogSourceDbModule (raw queries, no Drizzle modelling).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add server-side PostHog feature-flag evaluation to the API and wire three
high-leverage uses. The flags live in PostHog (project 127747), dormant and
fail-open, so this commit is a no-op until POSTHOG_PERSONAL_API_KEY is set and
a switch is flipped.
Phase 1 - upstream kill switches:
PostHogService gains local flag evaluation (onlyEvaluateLocally +
sendFeatureFlagEvents:false -> zero decode latency, no VIN leak) and
isSourceLive(). Guards at each source's decode entry (parts-catalogs, emex,
pl24 whole-source + per-brand via LEGACY_ARCH_SOURCE_TAG) let a flailing
upstream be disabled from the PostHog UI in ~5s instead of a
code-fix -> dev -> prod redeploy. Fail-open: any unresolved flag keeps the
source live, so a PostHog outage can never black out decoding.
Phase 2 - guarded rollout primitive:
isEnabled()/variant() with VIN bucketing, ready to ramp a new decode/parser
path 0->100% behind a decode-*-v2 flag (recipe in feature-flags-strategy.md).
Phase 4 - remote-config ops tuning:
prefetch-worker reads cfg-backfill-tuning to retune backfill
batchSize/maxBacklog/businessHoursOnly live; malformed/missing -> the
compiled-in constants.
POSTHOG_PERSONAL_API_KEY wired into the api + worker compose blocks (empty ->
flags inert, no added latency). Tests updated for the new constructor params.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two visible defects when a user picks a Ford model (variant selector step):
1. **N identical buttons.** Ford's `modelFamilyToModelList` gives every sub-catCode
the same caption — just the family name. parseFordCatCodesForFamily returned
`[{code:CBV,name:Kuga},{code:CBS,name:Kuga},{code:CTD,name:Kuga}]` for Kuga
2012-2020. The UI rendered three indistinguishable "Kuga" pills with no way
for the user to tell them apart. Same on Galaxy ("Galaxy"/"Galaxy"), every
multi-catCode family.
Fix: detect duplicate baseNames per family and graft a disambiguator —
year first (friendliest), catCode as a last resort. Result:
"Kuga (2013-2016)" / "Kuga (2016-2020)" / "Kuga (CTD)" — or, when year is
present even for unique entries, always include it for a uniform look.
2. **Section title was lying.** UI label was `catalog.fordVariant.modelYear`
= "Model Yılı" ("Model Year") but for Ford LEGACY these are catCodes
(generation identifiers), not years. Rename the key value to "Varyant" /
"Variant" — semantically correct for both Ford catCodes and Volvo years
(Volvo flows through the same selector and its codes ARE years; "Varyant"
covers both).
Cleared the affected Redis cache (`pl24:ford:config:*`) on dev so the next
hit fetches fresh disambiguated data; prod cache will roll over on its own
TTL after promote.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
These three brands rendered parts with NO schema illustration (and no hotspots):
schema_pics=0 vs 56/29/19 parts on dev. Root cause: extractPsaImageTicketUrl only
read imageViewerParamsUrl from the id="jsinitparams" data-params attribute (where
Ford/PSA/Opel/Volvo put it). Hyundai/Kia/Nissan's jsinitparams carries only a
localization dict ({"commonTxt":{"ok":"TAMAM"}}); their imageViewerParamsUrl lives
in a separate <script> JSON blob → extractor returned null → image pipeline skipped.
Fix: when the jsinitparams-attribute parse yields no URL, fall back to a whole-HTML
scan for "imageViewerParamsUrl":"…" (decoding &). Brand-agnostic and additive;
Ford/PSA/Opel/Volvo unchanged (attribute path still wins). Proven live via the de
client: Hyundai 600x820/3 hotspots, Kia 600x820/11, Nissan 1024x560/13 — all download OK.
Also: drop a pre-existing noParameterAssign in fetchP4Page (derive `account` const
from `accountParam` instead of reassigning the param). +3 extractor unit tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fiat catalog browse returned 0 models because the generic single-endpoint P5
model-list flow can't express Fiat's two-level hierarchy and the drill defaulted
to the tr account (Fiat is licensed only on de-708171).
- fetchVehicleList: dispatch fiatp_parts/fiatt_parts to new fetchFiatVehicleList,
which expands modelOverview (34 families) → models?modelFamily=N (model codes +
year ranges) into flat catalog vehicles whose catalogPath is the maingroups
endpoint. Verified live: 34 families → 123 models, end-to-end drill to parts+image.
- fetchMainGroups / fetchP5Restrictions: account-aware (resolveAccount → de + DE
proxy for Fiat; tr unchanged for every other P5 brand) so browse maingroups no
longer hit the tr demo/empty page.
- web: case-insensitive "/maingroup" gate so Fiat (lowercase /mdl/maingroups,
already a maingroups endpoint) skips the empty restriction selector and loads
categories directly.
- formatFiatYear: "(2016,2020)" → "2016-2020"; +unit tests.
Subgroups/parts/images already resolve account→de for Fiat (unchanged).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finding: /p5fiat/extern/vehicle/models IS the endpoint (de-licensed, demo:false) but
returns 'Cannot parse null string' regardless of manufacturer/make/brand param — needs
an undiscoverable param from the P5 SPA session (launchCatalog.do flow). Requires a real
PL24 web-UI network capture to resolve. Turkish Tofaş NM4/Egea is not in this catalog
anyway, so catalog-browse value is European-Fiat-only. Kept the generic P5 endpoint
self-discovery fallback. Fiat VIN-decode (European) is unaffected and still works.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ford parts are licensed only on the de account. The de drill chain is:
group.action (engine/transmission restriction nodes, HTML) → json-main-group.action
(maingroups JSON) → json-sub-group.action (subgroups JSON) → image-board.action (parts).
The existing json-sub-group handler only knew the Nissan shape ({caption,code,url});
Ford de subgroups are {caption,id,subheader,jsonUrl:null,url:null} — leaves carry no
URL. Add parseFordSubGroupsJson: skip subheader rows, and for each leaf construct the
illustration URL (subGroupId = mainGroupId+subheaderId, bomId = subGroupId+leafId →
image-board.action?bomId=…), which drills to parts via the existing image-board path.
Route Ford (fordp/fordt) to de alongside Hyundai/Kia/Nissan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tested Ford & Polestar on de: Ford returns a real variant tree but drilling can't
reach parts (needs dedicated Ford variant-drill logic); Polestar returns 0
categories (data gap). Revert them to tr; keep the verified Hyundai/Kia/Nissan
de-routing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verified on dev: the tr account serves NOT_LOGGED_IN_DEMO for Hyundai/Kia/Nissan
parts (browse-only), while the de (account 2) subscription is licensed and returns
real data (de: 266 Hyundai / 169 Kia models, non-demo, ~88/64 drill rows). Route
every P4 fetch for these brands to de in fetchP4Page (+ resolveAccount), and ensure
the de service token is authorized. Existing tr-coded catalog_vehicles must be
re-seeded so they carry de catalog codes (ops step on deploy).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Confirmed via dev probes: the de account is non-demo (licensed) for Hyundai
parts but does NOT have the EUR/TR-region catalogs our vehicles use, so the
fallback returned non-demo-but-empty and just doubled upstream load. Root cause
of empty Hyundai/Kia/Nissan parts is PL24 licensing/region (tr account not
licensed for these brands' parts) — a commercial issue, not code-fixable.
Keeping only the explanatory comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hyundai/Kia/Nissan parts-drilling returned a NOT_LOGGED_IN_DEMO page on the tr
account (license-gated), so live drills yielded 0 parts even though the tree
loads. After the tr re-auth retry still demos, try the de account once; if it
serves real (non-demo) content, use it. Tests whether de holds the brand license.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
emex's ssd tokens appear IP-bound: a rotating proxy sends each hop of a
decode→tree→drill→parts flow from a different exit IP, so emex rejects the
ssd and returns empty (the rotating-Floxy verify still seeded 0). Pin the
Floxy exit IP with a sticky session (password suffix
`_session-<id>_lifetime-300`); a rolling session id is reused across the
flow and rotates near expiry or after a transport failure (dead IP → fresh
one). lifetime via EMEX_FLOXY_LIFETIME (0 = rotating).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DataImpulse (74.81.81.81, random port 10001-10099) intermittently throws
connect-timeouts / resets, dropping real emex hits. Add a Floxy residential
fallback (residential.floxy.io:12321): fetchEmexHtml now runs a provider
schedule — 3 DataImpulse attempts (rotating port), then 2 Floxy attempts —
and only falls back on transport errors (a definitive HTTP answer like 404
still stops the schedule). On by default; endpoint/creds overridable via
EMEX_FLOXY_* env. Direct proxy-less last resort still gated by
EMEX_DIRECT_FALLBACK.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ford P4 catalog navigation is session-stateful: catCode group.action returns
the full Ford model picker (30 vehicle.action rows), and vehicle.action?modelFamily=X
loops back to the same picker; a bare group.action after select still returns the
picker (parsedGroups=0). Stateless GETs over a rotating proxy can't hold PL24's
server-side model selection → groups are unreachable (explains 0/57 categories_fetched
on prod). Needs session-continuity or browser-based navigation — tracked, not a quick fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
emex catalogs were seeded from the flat QuickGroups taxonomy (~351
unsearchable leaves on one page). Switch the seeder to the Vehicle.aspx
OEM-microfiche tree (top groups -> sub groups -> Unit.aspx leaves), which
also reaches units that have no QuickGroups gid (e.g. the torque
converter, OEM 5205547 on the Fiesta).
- emex-tree.parser.ts: parseVehicleTree / parseUnitLeaves — pure,
GET-based (per-href ssd, no browser/postback). Unit-tested vs fixtures.
- emex.service: fetchVehicleTree + drillVehicleNode; all 3 decode methods
now populate categoryTree; Unit.aspx leaves fetch parts directly.
- categories.service: revive the dormant tree-insert branch with group
markers (linkWid="emex-group" / hasSubgroups), add an emex getChildren
lazy-drill branch + emex-group parent guard, tighten leaf detection,
disambiguate colliding sub-group names.
- prefetch: emex Vehicle.aspx = parent to drill; Unit/QuickDetails = leaf.
No DDL and no frontend changes — the generic tree UI renders it like
pl24/pcat. Existing flat emex categories re-seed lazily once deleted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnosis captured: Ford catCode group.action returns a vehicle-selection
page (30 vehicle.action rows, no group rows) — needs an extra selection
level before groups. Polestar json-model-years returns an HTML landing page,
not JSON — needs a different navigation flow. Both tracked for follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fetchVehicleList fell back to /extern/vehicle/modelfamilies for any backend
not in BACKEND_MODEL_PATH; p5fiat (Fiat) isn't mapped, so it returned 0
models and Fiat seeded nothing (0 catalog_vehicles on dev+prod). When the
primary path yields nothing, try the other known P5 listing endpoints and
use the first that returns models, logging which one worked so it can be
pinned. Only runs on the empty path → mapped backends unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
parseFordGroupsFromHtml leaked header/breadcrumb links into the category
list as pseudo-categories that drill to nothing: Hyundai/Kia "Portal"
(portal.action), logout, and Nissan "Repair & Maintenance Information"
(external https://eu.nissan.biz/). Skip these in both the primary tr-row
parser and the <a href> fallback. Real categories (group.action /
group-detail.action / json-(main|sub)-group.action) are unaffected, and
Volvo's vin-group.action?...group1=... real categories are kept.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When PL24 can't identify a VIN, vin-group.action redirects to vehicle.action (a brand-wide model picker). The generic P4 parser collapsed that page's <title> to the brand name with a bogus VIN-char year, and doResolveVin's fastest-source-wins treated that truthy result as a definitive decode — masking the pcat/emex candidate fallback.
parseP4VehicleResponse now returns null when the extracted model is just the brand (or contains the VIN), so the pipeline falls back to pcat/emex instead of storing a fake vehicle. Live discovery: 7/11 Hyundai/Kia/Nissan VINs hit the picker (newer 2024+ / Nissan EU model-pick catalog); the 4 PL24-indexed VINs (GETZ 02, RIO 15/17/20) still decode. Adds pl24-ford-legacy.service.spec.ts; biome-formats two pre-existing lines in the touched file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
p5daimler vinfoBasic has no model_yili/üretim tarihi — only "Teslimat tarihi"
(delivery). The parser ignored it, so year fell to the VIN 10th-char and was
wrong (WDD…1… → 2001 even for 2009/2015/2018 cars). Add "teslimat_tarihi" to the
date-year fallback. Live-verified: C 180→2009, E 250 BlueTEC→2015, E 350→2018.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
"Seri" is "{line} {chassis} [{variant}]" and the trim already implies the line
("520i"→5, "X3 sDrive20i"→X3), so append only the chassis(+variant): "520i G30",
"X3 sDrive20i G01" (was "520i 5 G30" / the redundant "X3 sDrive20i X3 G01").
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
BMW (p5bmw) decode was weak: model was just the trim ("520i") with no
chassis/generation, body_type empty, because BMW has NO prNr segment and keeps
that data in distinct vinfoBasic labels the shared parser ignored. Live-verified
fields: "Seri"="5 G30" (chassis), "Karoseri"="Limousine" (body), "Tahrik"="RWD".
- model: fold the generation ("Seri"/"Model tanimi") into the model when present
and not already included → "520i 5 G30" (disambiguates E60/F10/G30 for parts).
- bodyType: fall back to vinfoBasic "Karoseri" when there's no prNr K8*.
- series: read "Seri"; driveType: read "Tahrik".
Year already comes from "Üretim tarihi" (P5 year fallback). VW/Audi (prNr) and
Mercedes ("Piyasa adı", no seri/karoseri) verified unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Fiat vinfoBasic record carries the row under `values` like p5vwag, but the
inner field names are swapped: p5fiat uses values.key=<label>,
values.description=<value> (vwag uses values.description=<label>,
values.value=<value>). The first pass read values.description as the label, so
model fell back to the numeric description code ("319") and year to 0. Detect
the shape via values.key. Live-verified: ZFA Panda→"Panda POP 1.2 8V 69CV 5M E6"
/2014, Grande Punto/2009, 500L/2015. Spec fixture corrected to the real shape.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
