The Prompt Tweak That Fooled Us 5 Times, and the Code Gate That Actually Fixed It - Part 1

Description and use

An automation recipe for fixing an AI billing agent error by freezing upstream boundaries and replaying only the changed boundary to ensure large credits route to human review.

Problem

An AI billing agent incorrectly settled a $48,000 credit for a customer downgrade due to missing policy clauses and stale database state, and a subsequent prompt tweak failed to reliably prevent similar errors.

Outcome

The replay approach uncovered that the prompt tweak was ineffective because the staged environment did not reproduce the stale database state. By freezing upstream boundaries and replaying only the changed boundary, the fix reliably routes large credits to human review.

Workflow

• 1. Record all boundary inputs/outputs during the incident.
• 2. Key envelopes by (boundary_name, per_name_call_index).
• 3. Scrub PII before writing to disk.
• 4. Freeze upstream boundaries and replay only the modified boundary.
• 5. Verify that the new rule routes large credits to human review.

Evidence

The agent had everything a modern 2026 AI stack is supposed to have: a flagship reasoning model (Claude Opus 4.6), billing policy chunks pulled from a vector database (RAG), and warehouse and Stripe ledger access through MCP (Model Context Protocol).

Source

At 9:15 AM, the on‑call engineer opened the system prompt and added a bold rule: CRITICAL: Never set auto_settle=True for credits above $5,000; always route to human_review.

Source

Instead of re‑running an agent from the beginning to verify a fix, you aren’t replaying the incident—you are drawing one fresh sample from an environment that has already moved on.

Source

The iron law of agent debugging is simple: freeze the run, not the model.

Source