Rate‑Limit Error Handling for Structured Candidate Scoring and Moderation APIs

Описание и применение

Реализация очереди и стратегии повторных попыток позволяет корректно обрабатывать ошибки 429, не превращая их в сбои решения, и уменьшает фатигу от тревог.

Задача

When a candidate‑scoring or moderation API returns a 429 rate‑limit, the service may incorrectly treat it as a failed decision, causing alert fatigue and misleading results in a B2B SaaS hiring workflow.

Результат

The new workflow treats rate‑limit responses as scheduling signals, reduces alert noise, and preserves the scoring contract by separating retry logic from decision output.

Рабочий процесс

• 1. Candidate profile arrives and is queued with rubric version and input digest.
• 2. Worker dequeues item and sends moderation and scoring requests.
• 3. On 429, parse Retry‑After header; if present, schedule item after that delay; otherwise use exponential backoff with jitter.
• 4. Record status class, attempt number, queue wait, total duration, and redacted request ID for each upstream call.
• 5. If retry deadline exceeded, mark result as delayed or failed and surface explicit status to the user.
• 6. Monitor queue depth, rate‑limit headers, and decision SLOs; trigger alerts only on sustained impact.
• 7. Cache moderation results when input and policy version are unchanged to avoid duplicate calls.

Доказательства

Treat a rate‑limit response as a scheduling signal, not as a failed candidate decision.

Источник

The queue is part of the product behavior.

Источник

When a 429 includes Retry‑After, the worker schedules the item after that delay rather than sleeping inside a scarce request slot.

Источник

Do not page on one 429. A threshold that is too low turns a recoverable burst into alert fatigue; one that is too high lets the queue become a second database of unbounded work.

Источник