My AI Agent Hit a Rate Limit at Midnight and Retried 11,000 Times. My "Smart" Retry Logic Was the Dumbest Code I've Shipped.
Описание и применение
Агент на Raspberry Pi, использующий простую экспоненциальную задержку, превратил одно ограничение скорости в 11 000 запросов, что привело к блокировке аккаунта. Внедрение APICaller с джиттером, бюджетом повторных попыток и переключателем цепи исправило проблему.
Задача
A Python agent on a Raspberry Pi retried API calls too aggressively, turning a single rate‑limit event into an 11,000‑request denial‑of‑service attack on its vendor.
Результат
The agent now limits retries globally, uses jittered backoff, and stops retrying when the vendor is rate‑limited, preventing repeated 11,000‑request spikes and avoiding account suspension.
Рабочий процесс
- 1. Detect rate‑limit (429) from any API call.
- 2. Use APICaller.call to retry with jittered exponential backoff.
- 3. Track failures and maintain a rolling retry budget of 20 attempts per minute.
- 4. Open circuit breaker after 10 consecutive failures for 5 minutes.
- 5. On sustained rate‑limit, write a cooling‑off file and exit with success code.
- 6. Configure systemd StartLimitIntervalSec/StartLimitBurst to avoid rapid restarts.
Доказательства
By 6 AM: ~11,000 requests, account flagged, three hours of my morning spent on a support call proving I wasn’t a crypto scraper.
Bug 1: Backoff without jitter. Every retry used exactly 2^attempt seconds.
- Дата записи: 2026-09-21T17:00:25.861Z