My Agent Gave a Refund on Day 47. My Policy Said 30. The Culprit Was 900 Words Down My Prompt.
Description and use
The refund system incorrectly processed requests after 30 days due to conflicting instructions in a long system prompt. Reordering rules, clarifying tone, and shortening the prompt raised policy compliance to 94 %.
Problem
Refunds were incorrectly processed beyond the 30‑day policy due to conflicting instructions in a long system prompt.
Outcome
Prompt reordering, tone clarification, and length reduction increased refund compliance from 23% to 94% for day‑47 cases, reducing policy violations and associated costs.
Workflow
- 1. Identify conflicting refund instructions in the system prompt.
- 2. Reorder prompt so the policy rule is last.
- 3. Clarify tone guidance to avoid overriding policy.
- 4. Reduce prompt length to avoid dilution of specificity.
- 5. Add structured decision output to enable measurement.
- 6. Run automated tests (20 runs per case) to evaluate compliance.
- 7. Adjust prompt based on results.
- 8. Deploy updated prompt to production.
Evidence
Line 203 beat line 41. By 160 lines and a lot of confident phrasing.
Identical text, three orderings, same 30 cases: Bottom of prompt 94%
Length diluted specificity, badly. 1,410 words 18% compliance
- Record date: 2026-11-01T02:58:55.466Z