Prevent Runaway LLM API Bills with an In-Process Circuit Breaker

说明与用途

通过实现本地电路断路器，可阻止无限循环并强制执行硬预算上限，防止出现意外的高额LLM‑API账单。

问题

Unexpected high API bills from runaway loops in autonomous agents, RAG pipelines, or LLM backends.

结果

Implementing LoopBreaker and BudgetGuard stops infinite loops and enforces a hard budget cap, eliminating unexpected high bills.

工作流

• Deploy agent with tool‑calling
• Detect unexpected JSON schema or exception
• Agent enters retry loop
• LoopBreaker hashes action signatures and raises RuntimeError after N repeats
• BudgetGuard calculates token cost from response.usage and raises Exception if budget exceeded
• Notify developers via webhook or log

证据

Every developer building autonomous agents, RAG pipelines, or LLM-powered backends shares the same quiet fear: waking up to an unexpected $300+ bill from OpenAI or Anthropic because of a runaway while-loop.

来源

Hard USD Budget Caps: An in‑memory ledger that calculates token cost in real time and throws an immediate BudgetExceededError if an operation would exceed your chosen threshold (e.g. $1.00 per session).

来源

Cyclic Loop Detection: A rolling action‑signature hash that identifies when an agent is repeating the exact same tool call or prompt sequence in a loop.

来源