One Project Ate the Shared Free Tier: A Per-Project Quota Pattern for LLM Gateways

说明与用途

多个项目共享单一LLM网关和1000万令牌配额，导致一个项目耗尽配额时其他项目失效。通过引入按项目计费、预检查、对账和排队机制，防止了此类问题。

问题

Multiple projects shared a single LLM gateway, API key, and 10‑million‑token allowance, causing one project’s batch job to exhaust the quota and break all others.

结果

The per‑project quota system prevented a single project from consuming the shared allowance, added graceful degradation, and ensured accurate usage tracking even after restarts.

工作流

• Introduce project ID in request header or path
• Define ProjectQuota objects with daily token and per‑minute limits
• Implement precheck to estimate tokens and reject if over budget
• Forward request to LLM if precheck passes
• After response, reconcile actual token usage and update counters
• Apply degradation ladder: warn at 80%, queue at 100%, reject over per‑minute cap
• Persist usage in SQLite and lazily reset daily counters on first request of new day

证据

Three projects shared one gateway, one API key, and one 10-million-token allowance.

来源

The pre-check prevented the common failure mode where a retry loop consumed the entire allowance before any monitoring alert could fire.

来源