The API Said 200. The Body Said 'Rate Limited'

Description and use

Client treated 200 as success, ignoring error payloads, leading to silent failures. Added envelope validation, error classification, and tests to catch 200 with error bodies.

Problem

Client treated HTTP 200 as success, ignoring error payloads that indicated rate limits, causing silent failures and missed retries.

Outcome

Client now correctly detects rate limit errors embedded in 200 responses, retries appropriately, and logs failures, preventing silent data loss.

Workflow

• 1. Send request and receive response
• 2. Check HTTP status code; if not 200, raise TransientError
• 3. Parse JSON body
• 4. Validate envelope: if "error" key present, classify error type
• 5. Raise specific exceptions (RateLimitError, ModelError, MalformedResponseError)
• 6. If no error, ensure "choices" field exists and is non-empty
• 7. Return completion text
• 8. Retry logic: on RateLimitError, apply exponential backoff and honor Retry-After; on other retryable errors, backoff with jitter; on non-retryable errors, fail fast

Evidence

My client checked response.status_code == 200 and moved on, so the errors flowed downstream as if they were completions

Source

The fix: validate the envelope, then classify the error

Source