feat(api): source-DB lookup-first for pcat/emex catalog fetches
Adds an optional local-dump lookup layer in front of the live PartsCatalogs and EMEX scrapes. When enabled, getCategoryWithPartsInner queries a Postgres (pcat) or MariaDB (emex) dump for the requested schema/group's parts and hotspots; on miss it falls through to the existing upstream call unchanged. Hits avoid the live API, its cooldown, and its rate-limits — direct DB latency. - New CatalogSourceDbModule with PcatSourceDbService + EmexSourceDbService (raw SQL, no Drizzle schema modeling — dump shapes are frozen snapshots). - pcat lookup keys on schema_images.schema_ext_id (the dump's column that matches sase's pcat groupId; observed ~7% hit rate on prod's 6596 unique groupIds, of which ~10% have schema_parts → ~3-5% net parts coverage). Joins schema_parts → parts directly; the dump's part_groups+part_group_items linkage covers 0 of our hits, so we skip that path entirely. - emex lookup uses (catalog_id, ssd) → vehicles.id then (vehicle_id, group_id) → vehicle_parts → parts + part_images. The ssd is already persisted into vehicles.rawData.ssd by the existing emex.mapper, no extra capture needed. Gated behind CATALOG_SOURCE_DB_ENABLED + PCAT_SOURCE_DB_URL / EMEX_SOURCE_DB_URL. All three default unset, so this commit is a no-op until prod env is configured. Adds mysql2 dep for the MariaDB client. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -58,6 +58,7 @@
|
||||
"drizzle-orm": "^0.41.0",
|
||||
"helmet": "^8.1.0",
|
||||
"ioredis": "^5.4.0",
|
||||
"mysql2": "^3.22.4",
|
||||
"openai": "^6.37.0",
|
||||
"postgres": "^3.4.0",
|
||||
"posthog-node": "^5.34.1",
|
||||
|
||||
Reference in New Issue
Block a user