My Agent Gave a Refund on Day 47. My Policy Said 30. The Culprit Was 900 Words Down My Prompt.
Описание и применение
Система возврата средств в e‑коммерции неправильно обрабатывала запросы после 30 дней из‑за конфликтующих инструкций в длинном системном запросе. Перестановка правил, уточнение тональности и сокращение длины запроса повысили соблюдение политики до 94 %.
Задача
Refunds were incorrectly processed beyond the 30‑day policy due to conflicting instructions in a long system prompt.
Результат
Prompt reordering, tone clarification, and length reduction increased refund compliance from 23% to 94% for day‑47 cases, reducing policy violations and associated costs.
Рабочий процесс
- 1. Identify conflicting refund instructions in the system prompt.
- 2. Reorder prompt so the policy rule is last.
- 3. Clarify tone guidance to avoid overriding policy.
- 4. Reduce prompt length to avoid dilution of specificity.
- 5. Add structured decision output to enable measurement.
- 6. Run automated tests (20 runs per case) to evaluate compliance.
- 7. Adjust prompt based on results.
- 8. Deploy updated prompt to production.
Доказательства
Line 203 beat line 41. By 160 lines and a lot of confident phrasing.
Identical text, three orderings, same 30 cases: Bottom of prompt 94%
Length diluted specificity, badly. 1,410 words 18% compliance
- Дата записи: 2026-11-01T02:58:55.466Z