The Prompt Tweak That Fooled Us 5 Times, and the Code Gate That Actually Fixed It - Part 1

Описание и применение

Автоматизированный рецепт исправления ошибки в биллинговом агенте: фиксируется состояние до и после, используется система воспроизведения с заморозкой входов/выходов, чтобы гарантировать, что правило о ручном обзоре применено.

Задача

An AI billing agent incorrectly settled a $48,000 credit for a customer downgrade due to missing policy clauses and stale database state, and a subsequent prompt tweak failed to reliably prevent similar errors.

Результат

The replay approach uncovered that the prompt tweak was ineffective because the staged environment did not reproduce the stale database state. By freezing upstream boundaries and replaying only the changed boundary, the fix reliably routes large credits to human review.

Рабочий процесс

• 1. Record all boundary inputs/outputs during the incident.
• 2. Key envelopes by (boundary_name, per_name_call_index).
• 3. Scrub PII before writing to disk.
• 4. Freeze upstream boundaries and replay only the modified boundary.
• 5. Verify that the new rule routes large credits to human review.

Доказательства

The agent had everything a modern 2026 AI stack is supposed to have: a flagship reasoning model (Claude Opus 4.6), billing policy chunks pulled from a vector database (RAG), and warehouse and Stripe ledger access through MCP (Model Context Protocol).

Источник

At 9:15 AM, the on‑call engineer opened the system prompt and added a bold rule: CRITICAL: Never set auto_settle=True for credits above $5,000; always route to human_review.

Источник

Instead of re‑running an agent from the beginning to verify a fix, you aren’t replaying the incident—you are drawing one fresh sample from an environment that has already moved on.

Источник

The iron law of agent debugging is simple: freeze the run, not the model.

Источник