I scored the tool descriptions of Chrome DevTools MCP and Playwright MCP. Both rarely say when to use a tool

说明与用途

浏览器自动化 MCP 服务器的工具描述常缺少使用时机和返回值信息，导致 AI 代理难以正确选择。Forecall 对描述进行评分，发现缺口；重写低分描述可提升分数，但可能降低某些模型的准确性。

问题

Tool descriptions in browser‑automation MCP servers often omit when to use a tool and what it returns, making it hard for AI agents to choose correctly.

结果

Tool descriptions in both MCP servers score poorly on usage guidance and return value. Adding concise, specific sentences can dramatically improve scores, but may negatively impact model accuracy for some agents.

工作流

• 1. Run each MCP server locally and capture tools/list JSON.
• 2. Use Forecall CLI (npx forecall lint) to score each tool on six parts: Purpose, When to use, Arguments, Return value, Constraints and side effects, Examples.
• 3. Analyze score distribution; identify gaps (missing when‑to‑use, return value).
• 4. Rewrite selected low‑scoring tool descriptions adding purpose, usage, constraints, return value and example.
• 5. Re‑score rewritten descriptions to confirm score improvement.
• 6. Test real models (Claude, GPT, Codex) on one‑step tasks to evaluate tool‑selection accuracy with original vs rewritten descriptions.
• 7. Record model performance and note any regressions.
• 8. Publish results and provide instructions for users to run Forecall on their own servers.
• 9. Encourage iterative rewriting and real‑model testing to close gaps.

证据

Chrome DevTools MCP, 24 of 30 tools never say when to use them; in Playwright MCP, 23 of 25.

来源

I rewrote drag's description using only facts from its schema. I added three things: when to use it (and how it differs from upload_file), what to do before calling it, and what it returns.

来源

With the descriptions as they are, no model picked a wrong tool: Claude Code with Fable 5.1, Sonnet 5.5 and Haiku 4.5 (24 of 24)... The only wrong picks, click instead of hover from two smaller Codex models, came from a task whose text was ambiguous.

来源