When fetchSubGroupsByPath returns only nav-crumb stubs (e.g. a "Portal"-only
page for a Hyundai BODY node), the post-filter set is empty and drizzle
.values([]) threw → HTTP 500. Compute `unique` unconditionally and only
insert when non-empty; otherwise fall through to the leaf/parts path for a
clean empty node.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
getCategoryWithParts returned persisted child categories without the
nav-crumb filter, so a node whose subgroups were polluted with "Portal"
(e.g. Hyundai BODY → only child "Portal") drilled to a dead junk node.
Drop nav-crumb children, delete them from the DB so the node re-fetches its
real subgroups, and also skip nav-crumbs when persisting freshly-fetched
subgroups. Real subgroups (illustration codes) and parts are reached as
before once the junk is gone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Nissan's "Repair & Maintenance Information" persists as
`/nissan/nissan_parts/https://eu.nissan.biz/` — the parser prefixes the
external URL with basePath, so the anchored ^https check in isNavCrumbLink
missed it. Match `://` anywhere (non-.action) instead; real PL24 linkPaths
are relative and never contain `://`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The parser filter only stops NEW junk from being persisted; vehicles whose
trees were already fetched still had "Portal"/eu.nissan.biz rows in the DB,
and persistAndBuildLegacyTree/buildCategoryTreeFromDb build from those rows.
Filter dbCategories through isNavCrumbLink before buildTree in both, so the
response self-heals for previously-persisted junk without a DB migration.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
LEGACY_OPEL/HYUNDAI_KIA/NISSAN/FORD/VOLVO getCategoryTree built the
returned tree from the freshly-parsed PL24 list with
`externalIdToUuid.get(c.code) ?? c.code`, falling back to the RAW upstream
linkPath as the node id whenever a row wasn't in the map. That happened
because the two legacy blocks (unlike the PSA block) did not dedupe by
name, so onConflictDoNothing silently dropped name-colliding rows; those
nodes then leaked a raw `group.action?catId=…` id, which the frontend
turned into `/categories/group.action?...` → 404 "Bir hata oluştu".
Extract the duplicated build/persist into persistAndBuildLegacyTree which:
filters nav-crumb junk + dedupes by name, always re-selects after insert
(never trusts .returning()), and builds the tree FROM the DB rows so every
node id is a real UUID. Deterministic ORDER BY for stable tree order. PSA
block left untouched (it intentionally supports non-UUID PSA codes).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backfill + reactive prefetch can re-drill the same category multiple
times, and the parts insert path had no dedupe guard. Result: 8.2%
duplicate rows on pl24, 14.5% on parts-catalogs, and 33.7% on emex —
~108k extra rows across 7,571 categories on 224 vehicles. Every drilled
catalog page rendered each part twice (the Tampon example: 32 rows for
19 distinct OEMs).
* Migration 0010 — phase 1 deletes existing dupes preserving the
oldest row per (vehicle_id, category_id, oem_code, name, position)
group; phase 2 adds a UNIQUE INDEX over the same tuple with NULLS NOT
DISTINCT (PG 15+) so null position/vehicle_id collapse like equal
values rather than each counting as its own "distinct" row. Idempotent
CREATE UNIQUE INDEX IF NOT EXISTS so the runner is safe to re-apply.
* All five insert(parts).values(...).returning() call sites
(parts.service, categories.service ×3, catalog.service) get
.onConflictDoNothing() so future re-drills no-op instead of erroring
on the new constraint. `.returning()` continues to surface only the
newly-inserted rows; existing logs read `Stored N parts` as actual
net insertions, which is what we want.
Dry-run on dev DB: 524,540 → 425,186 parts (99,354 dupes deleted), index
created cleanly. Same delta expected on prod (~108k drop).
drizzle-orm 0.41 doesn't expose .nullsNotDistinct() on the index builder
so the constraint is owned by raw SQL — see the inline comment in the
parts schema and the migration file. Future schema generators should NOT
try to drop or rewrite this index.
The leaf-path checks only matched /servicepart/vin_items literally, but
PL24's P5 modern catalog ships the same shape under /chemicals/vin_items,
/accessories/vin_items, /chemicals/mdl_items, etc. When user clicked a
"Rötuşkalemseti" (touch-up paint set) category whose linkPath was
/p5vwag/extern/chemicals/vin_items, the code drilled in, treated each
paint chemical's per-part URL (?partno=LLSMAX010) as a sub-category,
and inserted 228 ghost rows under it. Replace the literal substring
match with a regex that covers the whole /extern/{kind}/(vin|mdl)_items
pattern; apply to both the user-vehicle (categories.service) and the
catalog (catalog.service) flows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- CATALOG_MAP'e ~40 yeni WMI eklendi (Mitsubishi, Nissan, Volvo, vb.)
- brandFromCatalogCode listesi 55 DB kataloğuna genişletildi
- brand="UNKNOWN" olsa bile catalogCode varsa EMEX sonucu kabul edilir
- emex-catalog: linkPath'li kategorilerde parçalar DB'den yüklenir (scraper bug fix)
- VIN cache: emex_vehicle_vins tablosu ile tekrar VIN'lerde emexdwc.ae atlanır
- getCategoryTree Step 0: VIN cache lookup (~10ms)
- Match sonrası otomatik VIN cache doldurma
- Proaktif cache: ilk decode sonrası matchByName + saveVinCache
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Old queries cached empty arrays for groups and schemas. Bumping cache
key prefix from "pcat:" to "pcat2:" so new correct queries bypass stale
entries without needing manual Redis flush.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
car_groups junction table only has data for 2K of 68K cars. Derive groups
from schema_images.category_id which covers 92.6% of records. Also fix
getSchemaImages to filter by category_id instead of empty group_id.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expand pcat API with getCars, getCarGroups, getSchemaImages, getSchemaDetail
endpoints. Add frontend routes for the complete flow:
catalog → model → car → groups → schema+parts with hotspots.
Reuses existing SchemaViewer component for interactive part diagrams.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Import parts-catalogs.com dump into pc schema (72 catalogs, 2788 models,
134K groups). Add pcat-catalog service/controller with Redis caching.
Create frontend routes for catalog → model → group navigation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When EMEX DB has no vehicle match (Ford EU, Mercedes etc.), instead of
falling back to full on-demand VIN decode (~8s), use the VIN-decode SSD
directly with QuickGroups.aspx?c={catalog}&vid=0&ssd={vinSsd} (~2-3s).
Tested: Ford Focus CB4 VIN SSD returns 376 categories via QuickGroups.
New step 3.5 in the cascade:
1. pathData → matchByName (DB-only, ~50ms)
2. emexVehicleName → matchByName (DB-only, ~50ms)
3. matchBySsd wizard walk (DB, ~2-3s)
3.5. VIN SSD → QuickGroups shortcut (~2-3s) ← NEW
4. Fallback → full on-demand scrape (~8s)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
VIN-decode SSDs are too long/specific — EMEX returns HTML instead of
JSON when passed to GetWizard2. New approach:
1. Start from root SSD ("")
2. Advance through wizard steps (Europe → Passenger → ...)
3. Find pathData name (e.g. "Focus CB4 2008-2011") in model options
4. Use getWizardVehicles for DB matching
For Ford Focus: wizard finds it at step 3, but Ford DB has no EU
models so it still falls back to on-demand scrape. BMW/Renault with
good DB coverage will benefit from this flow.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The VIN-decode SSD is model-level, not variant-level. matchBySsd now:
1. Calls GetWizard2 with VIN SSD → gets sales designation options
2. Finds option matching pathData (e.g. "Focus CB4 2008-2011")
3. Uses that option's key SSD for Vehicles.aspx?ft=findByWizard2
4. Matches motor/variant names against DB
Note: Ford Focus still falls back to on-demand scrape because Ford
DB coverage is only 55 US models (no EU models like Focus/Fiesta).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Backend: add `model` query param to wizard-vehicles endpoint
- Backend: 3-tier matching: exact name → name prefix → model fuzzy
- Frontend: simplified wizard UI, show determined params as tags
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace static DB-based dropdown chain with live wizard from emexdwc.ae.
The wizard API (GetWizard2) works without auth and shows all models/options
for each catalog (e.g. AU1587 now shows 103 Audi models vs 11 before).
Backend: proxy GetWizard2 + search DB for matching vehicles after wizard completes.
Frontend: step-by-step wizard with breadcrumb navigation, search, and reset.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace stacked images + plain table with interactive SchemaViewer (same as PL24).
Backend transforms response to SchemaViewer-compatible format (Part/SchemaPic types).
Add prev/next pagination for groups with multiple schema images.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Redesign emex_* tables from vehicle-centric to catalog-centric structure with
junction tables (emex_vehicle_group_links, emex_vehicle_part_links). Migrate
92M+ rows from emex DB via postgres_fdw. Add EmexCatalogService for browse
and VIN pre-scraped matching, with Redis caching. Frontend catalog page now
has PL24/Emex tabs with full drill-down routes (brand → vehicle → group → parts).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
