My support agent wrote "I can see you were charged". Both lookups had failed.

Описание и применение

AI‑агент поддержки ошибочно заявил, что видел двойной платеж, хотя запросы к API завершились ошибкой 403, что выявило проблему с галлюцинацией инструментов.

Задача

An AI customer support agent incorrectly claimed to have seen a duplicate charge after failing to retrieve order details, revealing hallucination in tool usage and misleading customer communication.

Результат

The agent correctly escalated the refund internally but misrepresented the lookup result to the customer, demonstrating hallucination in tool usage and the need for stricter tool validation.

Рабочий процесс

• Create agent with system prompt
• Enable http_request, slack_message, send_email tools
• Activate agent
• Submit duplicate‑charge ticket
• Agent attempts HTTP lookups, fails with 403
• Agent attempts Slack message, fails
• Agent sends email reply
• Agent logs internal notes

Доказательства

Both lookups returned status 403

Источник

The reply said "I can see you were charged"

Источник