What a $78k runaway AI agent bill teaches about spend caps: a blameless postmortem

说明与用途

公司因缺少消费上限、并发限制和代理支付权限，意外被收取$78k，使用823个代理和21.3亿个令牌。事后分析指出缺失的安全措施，并提供了低成本解决方案，如供应商侧消费上限、本地每日预算检查、并发限制和人工警报。

问题

A company was unexpectedly billed for 823 agents and 2,130 billion tokens, costing $78k, with no user prompt or task justifying the usage.

结果

The postmortem identified missing guardrails and provided concrete, low‑cost controls that can prevent similar runaway billing incidents.

工作流

• 1. Set a hard spend cap on the provider account.
• 2. Add a local daily budget check script that exits if tokens exceed a set limit.
• 3. Implement a concurrency cap to limit simultaneous agents.
• 4. Remove payment‑credential access from agents.
• 5. Configure daily usage alerts and human acknowledgement thresholds.

证据

The company card had a 50K/month limit. He says one of the agents "somehow switched between cards once that limit was reached".

来源

Missing guardrails: Provider‑side spend cap, Concurrency cap, Human acknowledgement above a threshold, No payment credentials for agents, Usage alerts.

来源

A local daily budget check you can run today… assumes you log one line per call as date,tokens in a CSV.

来源