fix(api): defer rate-limited prefetch jobs instead of failing them #63

root · 2026-05-27T18:05:54+03:00

root commented

2026-05-27 18:05:54 +03:00

The worker threw a custom RateLimitError for active-user cooldown and off-hours
windows, but BullMQ treated that as a normal failure: it retried with the
queue's exponential backoff (ignoring retryAfterMs) and dropped the job to
"failed" after attempts:3. So during busy hours sub-jobs were discarded en
masse and the requested defer time (e.g. "wait until 09:00") never applied.

Convert RateLimitError into job.moveToDelayed(now + retryAfterMs, token) +
DelayedError — BullMQ's contract for "not done, not failed, retry later". This
does not consume an attempt and honours the exact delay, so cooldown jobs wait
~60-120s and off-hours jobs wait until the window reopens, then resume. Genuine
errors still fail/retry normally.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

The worker threw a custom RateLimitError for active-user cooldown and off-hours windows, but BullMQ treated that as a normal failure: it retried with the queue's exponential backoff (ignoring retryAfterMs) and dropped the job to "failed" after attempts:3. So during busy hours sub-jobs were discarded en masse and the requested defer time (e.g. "wait until 09:00") never applied. Convert RateLimitError into job.moveToDelayed(now + retryAfterMs, token) + DelayedError — BullMQ's contract for "not done, not failed, retry later". This does not consume an attempt and honours the exact delay, so cooldown jobs wait ~60-120s and off-hours jobs wait until the window reopens, then resume. Genuine errors still fail/retry normally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

root added 1 commit 2026-05-27 18:05:54 +03:00

fix(api): defer rate-limited prefetch jobs instead of failing them

QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

Details

c7e59b90e0

The worker threw a custom RateLimitError for active-user cooldown and off-hours
windows, but BullMQ treated that as a normal failure: it retried with the
queue's exponential backoff (ignoring retryAfterMs) and dropped the job to
"failed" after attempts:3. So during busy hours sub-jobs were discarded en
masse and the requested defer time (e.g. "wait until 09:00") never applied.

Convert RateLimitError into job.moveToDelayed(now + retryAfterMs, token) +
DelayedError — BullMQ's contract for "not done, not failed, retry later". This
does not consume an attempt and honours the exact delay, so cooldown jobs wait
~60-120s and off-hours jobs wait until the window reopens, then resume. Genuine
errors still fail/retry normally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

root merged commit 6e440f55de into main

2026-05-27 18:05:59 +03:00

root referenced this issue from a commit

2026-05-27 18:06:01 +03:00

Merge pull request 'fix(api): defer rate-limited prefetch jobs instead of failing them' (#63) from dev into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: root/sase.tr#63