Rate‑Limit Error Handling for Structured Candidate Scoring and Moderation APIs

Description and use

Implementing a bounded queue and retry strategy correctly handles 429 errors without treating them as decision failures, reducing alert fatigue.

Problem

When a candidate‑scoring or moderation API returns a 429 rate‑limit, the service may incorrectly treat it as a failed decision, causing alert fatigue and misleading results in a B2B SaaS hiring workflow.

Outcome

The new workflow treats rate‑limit responses as scheduling signals, reduces alert noise, and preserves the scoring contract by separating retry logic from decision output.

Workflow

• 1. Candidate profile arrives and is queued with rubric version and input digest.
• 2. Worker dequeues item and sends moderation and scoring requests.
• 3. On 429, parse Retry‑After header; if present, schedule item after that delay; otherwise use exponential backoff with jitter.
• 4. Record status class, attempt number, queue wait, total duration, and redacted request ID for each upstream call.
• 5. If retry deadline exceeded, mark result as delayed or failed and surface explicit status to the user.
• 6. Monitor queue depth, rate‑limit headers, and decision SLOs; trigger alerts only on sustained impact.
• 7. Cache moderation results when input and policy version are unchanged to avoid duplicate calls.

Evidence

Treat a rate‑limit response as a scheduling signal, not as a failed candidate decision.

Source

The queue is part of the product behavior.

Source

When a 429 includes Retry‑After, the worker schedules the item after that delay rather than sleeping inside a scarce request slot.

Source

Do not page on one 429. A threshold that is too low turns a recoverable burst into alert fatigue; one that is too high lets the queue become a second database of unbounded work.

Source