dev #92

root · 2026-06-03T20:22:02+03:00

root commented

2026-06-03 20:22:02 +03:00

No description provided.

root added 2 commits 2026-06-03 20:22:03 +03:00

perf(backfill): raise prefetch worker throughput, env-tunable (Tier 2) 48aea65b5c

After Tier 1 removed the self-throttle, the 5 jobs/min limiter + concurrency 1
became the bottleneck. Make throughput env-tunable so prod can ramp while
watching the fail rate:
- concurrency 1 -> 3 (PREFETCH_CONCURRENCY): parallelises emex/pl24 so a slow
  parts-catalogs job no longer head-of-line-blocks the queue.
- rate ceiling 5 -> 20 jobs/min (PREFETCH_RATE_MAX).
- parts-catalogs: drop the pathological cumulative index*20s enqueue delay (the
  Nth leaf of a vehicle waited N*20s); keep one per-job pace (PCAT_PACE_MS,
  default 15s, 0 to disable).
- PL24 09-18 scrape window now env-tunable (PREFETCH_PL24_START / _END).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

perf(backfill): Phase-1 residue exclusion — stop re-picking dead vehicles (Tier 3)

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

bb60a95ec3

Phase-1 of the backfill scan selects all zero-parts decoded vehicles every wave.
Genuine residue (VINs with no catalog data anywhere — model-indexed HKN,
EMEX-uncovered, etc.) stays zero-parts forever, so it filled the batch every
wave, re-attempting dead vehicles and starving the Phase-2 rolling rescan (its
cursor was stuck for a week).

Track a per-vehicle no-result counter (prefetch:noresult:<id>) incremented when a
prefetch attempt finishes with the vehicle still at zero parts (0 categories in
init, or 0 parts after the whole chain). tryPick skips vehicles past
PREFETCH_NORESULT_MAX (default 2) attempts; the counter has a TTL
(PREFETCH_NORESULT_TTL_DAYS, default 7) so a later catalog fix re-fills them.
Frees capacity for fillable vehicles and lets Phase-2 run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

root merged commit 9611d1c255 into main

2026-06-03 20:22:06 +03:00

root referenced this issue from a commit

2026-06-03 20:22:07 +03:00

Merge pull request 'dev' (#92) from dev into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: root/sase.tr#92