Your API Gateway Rate Limit Can't See the Agent Looping Your Search Tool

Описание и применение

Агент может зациковать на одном дорогим инструменте (например, web_search), создавая множество лервочных HTTP-запросов, которые выглядят нормально дляgateway-ограничения по протохчению. tool-throttle размещает токеч-банкировку на уровне вызова инструментов MCP, с отдельными квотами для каждого имени инструмента и шаблонов glob, что позволяет разработчику бюджетать расходы на конкретные инструменты, а не только на HTTP-слой.

Задача

An agent loops on a single expensive tool (e.g., web_search), making many valid HTTP requests that appear normal to an HTTP-layer gateway rate limiter, so the gateway never detects the abuse and the bill exceeds budget.

Результат

tool-throttle lets an agent developer budget per-tool call rates at the tool-call semantic layer instead of relying solely on HTTP-layer gateway limits, catching agents that loop on expensive tools while still allowing legitimate bursts through bounded queuing.

Рабочий процесс

• Install tool-throttle: pip install tool-throttle
• Create a ThrottlerConfig with default Quota and per_tool Quotas keyed by tool name or glob pattern, setting calls and per_seconds for each tool
• Set on_exceed to REJECT for fail-fast backoff or QUEUE for bounded-wait queuing
• Wrap the tool dispatcher with th.wrap(tool_name, mcp_client.call_tool)
• Call the wrapped tool; over quota raises ThrottledError with retry_after hint
• Inspect th.stats() to see allowed/rejected/queued/waited_seconds per tool
• Optionally save config as JSON via cfg.save('throttle.json') and use the CLI: tool-throttle check, tool-throttle acquire, or tool-throttle demo

Доказательства

An agent gets stuck. Not stuck like a crash — stuck like a loop: it calls web_search, gets results, doesn't like them, calls web_search again with a slightly different query. 10 times. 50 times. 400 times a minute.

Источник

The problem: gateway rate limits live at the HTTP layer. Agent abuse lives at the tool-call semantic layer. One HTTP request can fan out into many tool calls. One looping tool can spread its calls across many endpoints.

Источник

I built the fuse that belongs at the tool-call layer.

Источник

tool-throttle — a zero dependency Python package (stdlib only, Python >= 3.9) that puts a token bucket rate limiter on MCP tool calls, per tool name.

Источник

Quotas live in the language of the agent — tool names and glob patterns, with per-tool bursts — not in the language of HTTP.

Источник

It complements gateway rate limiting; it doesn't replace it. The gateway protects your infrastructure. tool-throttle protects your wallet from your own agent.

Источник