Your API Gateway Rate Limit Can't See the Agent Looping Your Search Tool

说明与用途

代理可能在一个昂贵的工具上循环，产生许多看似正常的HTTP请求，而网关速率限制无法检测到这种滥用。tool-throttle在MCP工具调用的语义层添加按工具名的令桶限

问题

An agent loops on a single expensive tool (e.g., web_search), making many valid HTTP requests that appear normal to an HTTP-layer gateway rate limiter, so the gateway never detects the abuse and the bill exceeds budget.

结果

tool-throttle lets an agent developer budget per-tool call rates at the tool-call semantic layer instead of relying solely on HTTP-layer gateway limits, catching agents that loop on expensive tools while still allowing legitimate bursts through bounded queuing.

工作流

• Install tool-throttle: pip install tool-throttle
• Create a ThrottlerConfig with default Quota and per_tool Quotas keyed by tool name or glob pattern, setting calls and per_seconds for each tool
• Set on_exceed to REJECT for fail-fast backoff or QUEUE for bounded-wait queuing
• Wrap the tool dispatcher with th.wrap(tool_name, mcp_client.call_tool)
• Call the wrapped tool; over quota raises ThrottledError with retry_after hint
• Inspect th.stats() to see allowed/rejected/queued/waited_seconds per tool
• Optionally save config as JSON via cfg.save('throttle.json') and use the CLI: tool-throttle check, tool-throttle acquire, or tool-throttle demo

证据

An agent gets stuck. Not stuck like a crash — stuck like a loop: it calls web_search, gets results, doesn't like them, calls web_search again with a slightly different query. 10 times. 50 times. 400 times a minute.

来源

The problem: gateway rate limits live at the HTTP layer. Agent abuse lives at the tool-call semantic layer. One HTTP request can fan out into many tool calls. One looping tool can spread its calls across many endpoints.

来源

I built the fuse that belongs at the tool-call layer.

来源

tool-throttle — a zero dependency Python package (stdlib only, Python >= 3.9) that puts a token bucket rate limiter on MCP tool calls, per tool name.

来源

Quotas live in the language of the agent — tool names and glob patterns, with per-tool bursts — not in the language of HTTP.

来源

It complements gateway rate limiting; it doesn't replace it. The gateway protects your infrastructure. tool-throttle protects your wallet from your own agent.

来源