One Project Ate the Shared Free Tier: A Per-Project Quota Pattern for LLM Gateways

Description and use

Multiple projects shared a single LLM gateway and a 10‑million‑token limit, causing failures when one project exhausted the quota. A per‑project quota system with pre‑check, reconciliation, and queuing prevented such incidents.

Problem

Multiple projects shared a single LLM gateway, API key, and 10‑million‑token allowance, causing one project’s batch job to exhaust the quota and break all others.

Outcome

The per‑project quota system prevented a single project from consuming the shared allowance, added graceful degradation, and ensured accurate usage tracking even after restarts.

Workflow

• Introduce project ID in request header or path
• Define ProjectQuota objects with daily token and per‑minute limits
• Implement precheck to estimate tokens and reject if over budget
• Forward request to LLM if precheck passes
• After response, reconcile actual token usage and update counters
• Apply degradation ladder: warn at 80%, queue at 100%, reject over per‑minute cap
• Persist usage in SQLite and lazily reset daily counters on first request of new day

Evidence

Three projects shared one gateway, one API key, and one 10-million-token allowance.

Source

The pre-check prevented the common failure mode where a retry loop consumed the entire allowance before any monitoring alert could fire.

Source