I built a router to cut my Claude Code bill, and prompt caching was the whole problem

Описание и применение

Создан роутер, который фиксирует уровень модели на уровне беседы, избегая кэш‑мисс и экономя до 90 % от стоимости работы с дорогой моделью Claude Code.

Задача

Prompt caching causes naive routing between cheap and frontier models to increase costs, negating savings.

Результат

The router achieved a $0.12 spend versus $1.23 frontier‑only on a sample run, saving 90% on that session and reducing overall frontier usage.

Рабочий процесс

• 1. Start a new conversation and send system prompt + first user message to router.
• 2. Router fingerprints the conversation and classifies difficulty (cheap, mid, frontier).
• 3. Router locks the conversation to the chosen tier.
• 4. Subsequent steps use the locked tier; if task becomes harder, a human can bump tier upward.
• 5. Downshifts only occur at cold boundaries (new conversation or reset).
• 6. Each request logs real cost and counterfactual frontier cost to a local ledger.
• 7. Review ledger for savings audit.

Доказательства

coding-agent-router savings report: $0.12 spent vs $1.23 frontier-only, 90% saved on that run

Источник

The design that actually works treats a conversation as the unit, not a request.