dev #79

root · 2026-06-02T02:02:23+03:00

root commented

2026-06-02 02:02:23 +03:00

No description provided.

root added 2 commits 2026-06-02 02:02:24 +03:00

fix(pl24): recover Volvo categories — stop vin-group.action?group1= shadowing 0f647b3931

Volvo (VIN-indexed legacy catalog) ships its real top groups as
vin-group.action?group1=… in the vin-group HTML. PL24's
json-vin-main-group.action endpoint now 404s, so decode falls back to the
HTML scrape — but parseP4NavigationCategories AND the seed-time NAV_CRUMB
filter both blanket-exclude vin-group.action, dropping every real Volvo
group → 0 categories. Exclude vin-group.action as a crumb only when it
lacks group1=. Confirmed upstream: 7 real groups (Frenler, Elektrik
sistemi, …) present in the HTML for YV1AS7050A1118639.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

feat(backfill): widen scrape window for pcat + double batch size

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

316a5e014c

Backfill was sweeping only 20 vehicles/hour and skipping pcat outside
09:00–19:00 Istanbul, leaving the catalog backlog (115 pl24 + 63 pcat +
27 emex zero-parts vehicles) crawling forward at ~528 parts/day and the
biggest user vehicle (Toyota Corolla 2026) unchanged across 24h.

Two related changes — both unblocked by the Redis-persisted warm JWT pool
(aa4d055 + dcf7e06) which keeps pcat captures alive 24/7 instead of
needing the 09:00–19:00 office-hours assumption:

* isWithinTimeWindow: parts-catalogs no longer gated — the warm pool +
  Redis hydration cover the cold-start case the old office-hours rule
  was working around. PL24 keeps its 09:00–18:00 window because the
  upstream rate-limit is still tighter outside it.
* BACKFILL_BATCH_SIZE 20 → 40 — twice as many vehicles per wave, still
  protected by MAX_BACKLOG=1000 self-throttle and per-source cooldown
  (prefetch:activity:<source>) so live-user traffic still gets priority.

Combined effect: pcat goes from ~10h/day to 24h/day, batch doubles —
roughly 3× backfill throughput. Worst-case proxy spend tracked by the
DataImpulse daily cap; if a wave saturates upstream the rate-limit
handler (c7e59b9) defers without burning attempts.

root added 1 commit 2026-06-02 02:08:13 +03:00

fix(pl24): Volvo subgroup drilling via vin-group.action HTML

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

d792bf0efb

Volvo's VIN catalog is a 3-level vin-group.action?group1=…[&group2=…] HTML
tree; PL24's json-vin-*-group.action JSON endpoints now 404. Add HTML
subgroup extraction (keep links one group-level deeper than the current
path) in ford-legacy fetchSubGroupsByPath, and route vin-group.action?group1=
nodes through getChildren (drill-first, fall back to parts) in
getCategoryWithParts. Recovers Volvo group1→group2 navigation. Leaf parts
handled separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

root added 1 commit 2026-06-02 02:08:58 +03:00

fix(parts): dedupe ingest path + clean ~108k duplicate rows

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

10dcf2d6ae

Backfill + reactive prefetch can re-drill the same category multiple
times, and the parts insert path had no dedupe guard. Result: 8.2%
duplicate rows on pl24, 14.5% on parts-catalogs, and 33.7% on emex —
~108k extra rows across 7,571 categories on 224 vehicles. Every drilled
catalog page rendered each part twice (the Tampon example: 32 rows for
19 distinct OEMs).

* Migration 0010 — phase 1 deletes existing dupes preserving the
  oldest row per (vehicle_id, category_id, oem_code, name, position)
  group; phase 2 adds a UNIQUE INDEX over the same tuple with NULLS NOT
  DISTINCT (PG 15+) so null position/vehicle_id collapse like equal
  values rather than each counting as its own "distinct" row. Idempotent
  CREATE UNIQUE INDEX IF NOT EXISTS so the runner is safe to re-apply.
* All five insert(parts).values(...).returning() call sites
  (parts.service, categories.service ×3, catalog.service) get
  .onConflictDoNothing() so future re-drills no-op instead of erroring
  on the new constraint. `.returning()` continues to surface only the
  newly-inserted rows; existing logs read `Stored N parts` as actual
  net insertions, which is what we want.

Dry-run on dev DB: 524,540 → 425,186 parts (99,354 dupes deleted), index
created cleanly. Same delta expected on prod (~108k drop).

drizzle-orm 0.41 doesn't expose .nullsNotDistinct() on the index builder
so the constraint is owned by raw SQL — see the inline comment in the
parts schema and the migration file. Future schema generators should NOT
try to drop or rewrite this index.

root added 1 commit 2026-06-02 02:11:38 +03:00

fix(ui): repair Turkish labels + hero preview model name

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

497aba24ab

Two cosmetic fixes the founder flagged in the same browse session:

* parts-panel.tsx — the catalog parts table was missing Turkish
  diacritics on every static label, while the actual part names rendered
  correctly. Looked like an encoding bug; was actually hard-coded ASCII
  strings. Repaired: Parcalar→Parçalar, parca listeleniyor→parça
  listeleniyor, Bu kategori icin parca bulunamadi→Bu kategori için
  parça bulunamadı, Yukleniyor→Yükleniyor, Parca Adi→Parça Adı,
  Geri don→Geri dön. OEM Kodu was already correct.

* index.tsx hero preview — pl24 ships model names with trailing
  duplicates ("Golf 1,6 GOLF"), and the card stitched year right next
  to the engine "(2003 4 silindirli…)" with no clear separator. Now:
  cleanModelName collapses the dup ("Golf 1,6"), year wraps in
  parentheses as a softer secondary element, and the visual flex gap
  guarantees breathing room before the engine pill.

root added 1 commit 2026-06-02 02:13:01 +03:00

fix(pl24): prefix basePath for relative P4 hrefs in fetchP4Page

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

7208069a80

Volvo vin-group.action categories store hrefs relative to the catalog dir
(e.g. "vin-group.action?group1=…"). fetchP4Page did `${baseUrl}${url}`,
collapsing to "partslink24.comvin-group.action" → ENOTFOUND. Prefix the
service basePath when the path is relative. Fixes Volvo subgroup drilling
for existing (relative) stored linkPaths without a re-decode.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

root added 1 commit 2026-06-02 02:18:06 +03:00

fix(pl24): demo-page retry in fetchP4Page + Volvo drill diagnostics

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

f5c3198a6e

PL24 serves a stripped NOT_LOGGED_IN_DEMO page (no groups/parts) when the
service token is stale. decodeVinForService retries on this, but drill paths
(fetchSubGroupsByPath/fetchPartsByPath) reach upstream only via fetchP4Page,
which didn't — so Volvo subgroup drilling parsed empty demo pages. Retry once
with fresh auth on a demo page. Adds a Volvo-drill diagnostic log.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

root added 1 commit 2026-06-02 02:25:01 +03:00

fix(pl24): Volvo drill — keep openVinDialog=false links + image-board leaves

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

c46695d320

Two bugs in the vin-group.action subgroup scrape: (1) the filter dropped any
href containing "openVinDialog", but the real sub-group links carry
openVinDialog=false (only the VIN-dialog crumb is =true) — so every child was
discarded; (2) the deepest group level lists its illustration leaves as
vin-image-board.action links, which weren't extracted. Now match both deeper
vin-group.action?groupN= and vin-image-board.action anchors, and only drop the
openVinDialog=true crumb. Completes Volvo group1→group2→illustration→parts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

root added 1 commit 2026-06-02 02:30:57 +03:00

fix(pl24): Volvo vin-image-board BOM parts + decode subgroup name entities

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

00f7edbc15

Volvo's vin-image-board.action ships its BOM inline as partno= tc-data-row
rows (no pncHierCode / json-vin-bom-detail), so the Ford VIN-BOM parser
returned 0 parts. Fall back to parsePsaBomParts (partno= rows) when no pnc
rows are found. Also decode HTML entities (&Ouml; &quot; …) in scraped Volvo
subgroup names. Completes the Volvo chain: group1→group2→illustration→parts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

root merged commit ed01ee44d2 into main

2026-06-02 02:37:52 +03:00

root referenced this issue from a commit

2026-06-02 02:37:54 +03:00

Merge pull request 'dev' (#79) from dev into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: root/sase.tr#79