Anthropic SDK max_retries Stacked on My Retries: 36 Calls per Job

Описание и применение

Многократные слои повторных попыток привели к 36 HTTP‑запросам на одну ошибку, перегрузив API и прерывая живые сессии. Устранение лишних ретраев и разделение клиентов по типу трафика восстановили нормальную работу.

Задача

Multiple retry layers multiplied, causing 36 HTTP attempts per failed call and a retry storm that overwhelmed the API and broke live traffic.

Результат

By eliminating redundant retries and separating traffic classes, the retry storm was prevented, live traffic remained unaffected, and background job throughput returned to normal.

Рабочий процесс

• 1. Identify all retry layers: SDK, tenacity, RQ job retry. 2. Remove tenacity wrapper around API calls. 3. Create separate Anthropic clients for background and live traffic. 4. Set SDK max_retries to 0 for background client or keep default for live client. 5. Cap background job queue size and add rate limiting. 6. Add httpx event hook to count real HTTP requests. 7. Monitor ratio of HTTP requests to app calls. 8. Adjust retry strategy based on observed ratios.

Доказательства

The Anthropic Python SDK retries a failed request twice by default, so one client.messages.create() can become 3 HTTP requests.

Источник

If you wrap messages.create() in tenacity, and your job queue also retries, the attempts multiply, not add. Mine was 3 × 4 × 3 = 36.

Источник

140 report jobs, which should have been about 420 requests (3 calls each) – 1,302 calls to call_claude() according to my app logs – 3,904 actual HTTP requests according to the httpx hook.

Источник