I Built an AI Agent That Worked in the Demo and Failed on Its First Real User

Description and use

<p>The demo went perfectly. Stakeholders nodded, someone called it magic, and we released the agent to a small group of real users the following Monday. Within an hour, it had issued a refund against the wrong order, called the same tool eleven times in a row, and replied to a customer in a language they never wrote in.</p> <p>Nothing was wrong with the model. What failed was everything around it. This post covers what broke, why the demo hid it, and the specific changes that made the agent survive contact with real people.</p> <h2> The Setup </h2> <p>The agent handled support tickets for an e

Problem

<p>The demo went perfectly. Stakeholders nodded, someone called it magic, and we released the agent to a small group of real users the following Monday. Within an hour, it had issued a refund against the wrong order, called the same tool eleven times in a row, and replied to a customer in a language they never wrote in.</p> <p>Nothing was wrong with the model. What failed was everything around it. This post covers what broke, why the demo hid it, and the specific changes that made the agent survive contact with real people.</p> <h2> The Setup </h2> <p>The agent handled support tickets for an e

Outcome

<p>The demo went perfectly. Stakeholders nodded, someone called it magic, and we released the agent to a small group of real users the following Monday. Within an hour, it had issued a refund against the wrong order, called the same tool eleven times in a row, and replied to a customer in a language they never wrote in.</p> <p>Nothing was wrong with the model. What failed was everything around it. This post covers what broke, why the demo hid it, and the specific changes that made the agent survive contact with real people.</p> <h2> The Setup </h2> <p>The agent handled support tickets for an e

Evidence

I Built an AI Agent That Worked in the Demo and Failed on Its First Real User

Source