sase.tr

Files

Semih Yesilyurt 48aea65b5c perf(backfill): raise prefetch worker throughput, env-tunable (Tier 2)

After Tier 1 removed the self-throttle, the 5 jobs/min limiter + concurrency 1
became the bottleneck. Make throughput env-tunable so prod can ramp while
watching the fail rate:
- concurrency 1 -> 3 (PREFETCH_CONCURRENCY): parallelises emex/pl24 so a slow
  parts-catalogs job no longer head-of-line-blocks the queue.
- rate ceiling 5 -> 20 jobs/min (PREFETCH_RATE_MAX).
- parts-catalogs: drop the pathological cumulative index*20s enqueue delay (the
  Nth leaf of a vehicle waited N*20s); keep one per-job pace (PCAT_PACE_MS,
  default 15s, 0 to disable).
- PL24 09-18 scrape window now env-tunable (PREFETCH_PL24_START / _END).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-03 18:43:10 +03:00