The One Rule a System Prompt Could Not Hold

Description and use

<p>We build Dograh, an open source voice agent platform, and the tightest requirement anyone has handed us arrived from a bank fraud team as a single sentence. The agent must never ask a customer to reveal their card number.</p> <p>Easy. Put it in the system prompt. Never request the full card number, never ask for a passcode, never ask for a date of birth. Run the evals. It passes. It keeps passing through staging, through the pilot, through a few thousand calls.</p> <p>Then a customer answers the confirm-or-deny question with something slightly sideways, and the agent says: "no problem, can

Problem

<p>We build Dograh, an open source voice agent platform, and the tightest requirement anyone has handed us arrived from a bank fraud team as a single sentence. The agent must never ask a customer to reveal their card number.</p> <p>Easy. Put it in the system prompt. Never request the full card number, never ask for a passcode, never ask for a date of birth. Run the evals. It passes. It keeps passing through staging, through the pilot, through a few thousand calls.</p> <p>Then a customer answers the confirm-or-deny question with something slightly sideways, and the agent says: "no problem, can

Outcome

<p>We build Dograh, an open source voice agent platform, and the tightest requirement anyone has handed us arrived from a bank fraud team as a single sentence. The agent must never ask a customer to reveal their card number.</p> <p>Easy. Put it in the system prompt. Never request the full card number, never ask for a passcode, never ask for a date of birth. Run the evals. It passes. It keeps passing through staging, through the pilot, through a few thousand calls.</p> <p>Then a customer answers the confirm-or-deny question with something slightly sideways, and the agent says: "no problem, can

Evidence

The One Rule a System Prompt Could Not Hold

Source