Postmortem: The Free Tier Wasn't the Problem — Our Guardrails Were

Description and use

Adding concurrency limits, retry caps, and a daily budget eliminated CI pipeline failures, improved reliability, and reduced latency.

Problem

CI pipeline failed because free tier guardrails were missing, causing retry storms, OOM, and incomplete reviews.

Outcome

Guardrails reduced failure rate from 39% to 6%, eliminated OOM and retry storms, and improved latency.

Workflow

• 1. Identify free tier usage in CI pipeline
• 2. Build GuardedClient with concurrency, retry, budget limits
• 3. Wrap API calls with GuardedClient
• 4. Add endpoint routing decision matrix to route critical tasks to paid tier
• 5. Monitor success rate, latency, and budget usage
• 6. Adjust parameters (max_concurrency, max_retries, daily_budget) as needed

Evidence

The first rate limit responses arrived. HTTP 429 mixed with HTTP 200 in the logs.

Source

Success rate: 94% (up from 61%)

Source

No OOM or retry storm

Source