The API Said 200. The Body Said 'Rate Limited'

Описание и применение

Клиент, принимающий 200 как успех, игнорировал ошибки в теле ответа, что приводило к неотловленным сбоям. Добавлен контроль над телом, классификация ошибок и тесты, имитирующие 200 с ошибкой.

Задача

Client treated HTTP 200 as success, ignoring error payloads that indicated rate limits, causing silent failures and missed retries.

Результат

Client now correctly detects rate limit errors embedded in 200 responses, retries appropriately, and logs failures, preventing silent data loss.

Рабочий процесс

• 1. Send request and receive response
• 2. Check HTTP status code; if not 200, raise TransientError
• 3. Parse JSON body
• 4. Validate envelope: if "error" key present, classify error type
• 5. Raise specific exceptions (RateLimitError, ModelError, MalformedResponseError)
• 6. If no error, ensure "choices" field exists and is non-empty
• 7. Return completion text
• 8. Retry logic: on RateLimitError, apply exponential backoff and honor Retry-After; on other retryable errors, backoff with jitter; on non-retryable errors, fail fast

Доказательства

My client checked response.status_code == 200 and moved on, so the errors flowed downstream as if they were completions

Источник

The fix: validate the envelope, then classify the error

Источник