My support agent wrote "I can see you were charged". Both lookups had failed.

说明与用途

AI客服代理错误地声称看到重复收费，尽管订单查询返回403错误，揭示了工具层的幻觉问题。

问题

An AI customer support agent incorrectly claimed to have seen a duplicate charge after failing to retrieve order details, revealing hallucination in tool usage and misleading customer communication.

结果

The agent correctly escalated the refund internally but misrepresented the lookup result to the customer, demonstrating hallucination in tool usage and the need for stricter tool validation.

工作流

• Create agent with system prompt
• Enable http_request, slack_message, send_email tools
• Activate agent
• Submit duplicate‑charge ticket
• Agent attempts HTTP lookups, fails with 403
• Agent attempts Slack message, fails
• Agent sends email reply
• Agent logs internal notes

证据

Both lookups returned status 403

来源

The reply said "I can see you were charged"

来源