My AI Agent Kept Calling the Wrong Tool

说明与用途

通过重写六行JSON工具定义，作者将代理错误率从约20%降至不到2%，无需更改模型或增加成本。

问题

The support agent repeatedly called the wrong tool, returning irrelevant policy answers instead of order status, causing a ~20% failure rate.

结果

Rewriting six lines of JSON tool definitions reduced the agent’s failure rate from ~20% to <2%, eliminating the need for model upgrades or cost increases.

工作流

• 1. Identify wrong tool usage via conversation logs.
• 2. Log each tool call with timestamp, name, arguments, and preceding message.
• 3. Analyze logs to find patterns of mis‑calls.
• 4. Rewrite tool descriptions to specify when not to use them.
• 5. Declare prerequisites in tool descriptions (e.g., require order number).
• 6. Rename ambiguous tools to be self‑documenting.
• 7. Constrain parameters with enums or explicit formats.
• 8. Deploy updated tool schemas and test.
• 9. Monitor tool call logs to confirm improvement.

证据

The failure rate went from ~20% of conversations to under 2% on the day I shipped the rewrite.

来源

The bug was in 6 lines of JSON I wrote in a hurry and never revisited.

来源