How many tools should an MCP server have? I measured 5 to 100 in Claude Code, Codex and Gemini

Описание и применение

Все тестируемые количества инструментов (от 5 до 100) обеспечили 100 % точность, но стоимость и контекст зависят от того, как инструменты загружаются клиентом. Включённый поиск инструментов в Claude Code снижает токены при большом количестве, но добавляет дополнительный вызов. Группировка инструментов экономит только при полной загрузке схем.

Задача

Determining the optimal number of tools for an MCP server to balance accuracy, cost, and context across different LLM clients.

Результат

Accuracy remained 100 % from 5 to 100 tools; cost differences were driven by tool loading strategy (search on/off, eager vs deferred) rather than tool count. Tool search in Claude Code reduced input tokens above ~19 tools but increased cost slightly due to an extra call. Grouping tools saved cost only when all schemas were eagerly loaded.

Рабочий процесс

• 1. Reviewed 91 MCP server design sources to gather existing recommendations.
• 2. Built a fake MCP server with 100 realistic file‑handling tools, including look‑alike and vague‑named variants.
• 3. Ran 18 tasks (10 ambiguous) across tool counts 5, 10, 20, 30, 50, 75, 100, each twice, with three clients.
• 4. Collected input token counts for first call and per run under different loading modes.
• 5. Analyzed accuracy, token cost, and impact of tool search, grouping, and naming.
• 6. Summarized findings and provided actionable guidance.

Доказательства

Every run with separate tools was fully correct, from 5 to 100 tools, in all three clients and in both Claude Code modes.

Tool search is the switch that matters in Claude Code. It's on by default: only tool names go into the prompt, and the model loads the schemas it needs with one ToolSearch call.

With every schema loaded (Claude Code, search off), 7 grouped tools cost what 14 separate ones do, at 100% accuracy.

Names that misled most often: "3 JPGs into one PDF" led to combine_files, which actually merges PDFs; "shrink a PNG" led to optimize_image; and pptx and docx to PDF got mixed up between convert_file and convert_document.