Phase-1 of the backfill scan selects all zero-parts decoded vehicles every wave.
Genuine residue (VINs with no catalog data anywhere — model-indexed HKN,
EMEX-uncovered, etc.) stays zero-parts forever, so it filled the batch every
wave, re-attempting dead vehicles and starving the Phase-2 rolling rescan (its
cursor was stuck for a week).
Track a per-vehicle no-result counter (prefetch:noresult:<id>) incremented when a
prefetch attempt finishes with the vehicle still at zero parts (0 categories in
init, or 0 parts after the whole chain). tryPick skips vehicles past
PREFETCH_NORESULT_MAX (default 2) attempts; the counter has a TTL
(PREFETCH_NORESULT_TTL_DAYS, default 7) so a later catalog fix re-fills them.
Frees capacity for fillable vehicles and lets Phase-2 run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After Tier 1 removed the self-throttle, the 5 jobs/min limiter + concurrency 1
became the bottleneck. Make throughput env-tunable so prod can ramp while
watching the fail rate:
- concurrency 1 -> 3 (PREFETCH_CONCURRENCY): parallelises emex/pl24 so a slow
parts-catalogs job no longer head-of-line-blocks the queue.
- rate ceiling 5 -> 20 jobs/min (PREFETCH_RATE_MAX).
- parts-catalogs: drop the pathological cumulative index*20s enqueue delay (the
Nth leaf of a vehicle waited N*20s); keep one per-job pace (PCAT_PACE_MS,
default 15s, 0 to disable).
- PL24 09-18 scrape window now env-tunable (PREFETCH_PL24_START / _END).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
