Rate‑Limit Error Handling for Structured Candidate Scoring and Moderation APIs

说明与用途

通过实现有界队列和重试策略，正确处理 429 错误，避免将其视为决策失败，从而降低警报疲劳。

问题

When a candidate‑scoring or moderation API returns a 429 rate‑limit, the service may incorrectly treat it as a failed decision, causing alert fatigue and misleading results in a B2B SaaS hiring workflow.

结果

The new workflow treats rate‑limit responses as scheduling signals, reduces alert noise, and preserves the scoring contract by separating retry logic from decision output.

工作流

• 1. Candidate profile arrives and is queued with rubric version and input digest.
• 2. Worker dequeues item and sends moderation and scoring requests.
• 3. On 429, parse Retry‑After header; if present, schedule item after that delay; otherwise use exponential backoff with jitter.
• 4. Record status class, attempt number, queue wait, total duration, and redacted request ID for each upstream call.
• 5. If retry deadline exceeded, mark result as delayed or failed and surface explicit status to the user.
• 6. Monitor queue depth, rate‑limit headers, and decision SLOs; trigger alerts only on sustained impact.
• 7. Cache moderation results when input and policy version are unchanged to avoid duplicate calls.

证据

Treat a rate‑limit response as a scheduling signal, not as a failed candidate decision.

来源

The queue is part of the product behavior.

来源

When a 429 includes Retry‑After, the worker schedules the item after that delay rather than sleeping inside a scarce request slot.

来源

Do not page on one 429. A threshold that is too low turns a recoverable burst into alert fatigue; one that is too high lets the queue become a second database of unbounded work.

来源