feat(api): source-DB lookup-first for pcat/emex catalog fetches

Adds an optional local-dump lookup layer in front of the live PartsCatalogs and
EMEX scrapes. When enabled, getCategoryWithPartsInner queries a Postgres
(pcat) or MariaDB (emex) dump for the requested schema/group's parts and
hotspots; on miss it falls through to the existing upstream call unchanged.
Hits avoid the live API, its cooldown, and its rate-limits — direct DB latency.

- New CatalogSourceDbModule with PcatSourceDbService + EmexSourceDbService
  (raw SQL, no Drizzle schema modeling — dump shapes are frozen snapshots).
- pcat lookup keys on schema_images.schema_ext_id (the dump's column that
  matches sase's pcat groupId; observed ~7% hit rate on prod's 6596 unique
  groupIds, of which ~10% have schema_parts → ~3-5% net parts coverage).
  Joins schema_parts → parts directly; the dump's part_groups+part_group_items
  linkage covers 0 of our hits, so we skip that path entirely.
- emex lookup uses (catalog_id, ssd) → vehicles.id then (vehicle_id, group_id)
  → vehicle_parts → parts + part_images. The ssd is already persisted into
  vehicles.rawData.ssd by the existing emex.mapper, no extra capture needed.

Gated behind CATALOG_SOURCE_DB_ENABLED + PCAT_SOURCE_DB_URL / EMEX_SOURCE_DB_URL.
All three default unset, so this commit is a no-op until prod env is configured.
Adds mysql2 dep for the MariaDB client.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-01 00:35:45 +03:00
parent d9b14af96d
commit 4724a71113
10 changed files with 557 additions and 19 deletions

View File

@@ -58,6 +58,7 @@
"drizzle-orm": "^0.41.0",
"helmet": "^8.1.0",
"ioredis": "^5.4.0",
"mysql2": "^3.22.4",
"openai": "^6.37.0",
"postgres": "^3.4.0",
"posthog-node": "^5.34.1",