Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

My Agent Gave a Refund on Day 47. My Policy Said 30. The Culprit Was 900 Words Down My Prompt.

Description and use

The refund system incorrectly processed requests after 30 days due to conflicting instructions in a long system prompt. Reordering rules, clarifying tone, and shortening the prompt raised policy compliance to 94 %.

Problem

Refunds were incorrectly processed beyond the 30‑day policy due to conflicting instructions in a long system prompt.

Outcome

Prompt reordering, tone clarification, and length reduction increased refund compliance from 23% to 94% for day‑47 cases, reducing policy violations and associated costs.

Workflow

  • 1. Identify conflicting refund instructions in the system prompt.
  • 2. Reorder prompt so the policy rule is last.
  • 3. Clarify tone guidance to avoid overriding policy.
  • 4. Reduce prompt length to avoid dilution of specificity.
  • 5. Add structured decision output to enable measurement.
  • 6. Run automated tests (20 runs per case) to evaluate compliance.
  • 7. Adjust prompt based on results.
  • 8. Deploy updated prompt to production.

Evidence

Line 203 beat line 41. By 160 lines and a lot of confident phrasing.

Source

Identical text, three orderings, same 30 cases: Bottom of prompt 94%

Source

Length diluted specificity, badly. 1,410 words 18% compliance

Source

  • Record date: 2026-11-01T02:58:55.466Z