I built a router to cut my Claude Code bill, and prompt caching was the whole problem

Description and use

A router locks the model tier per conversation, preventing cache misses and saving up to 90 % compared to always using the expensive Claude Code model.

Problem

Prompt caching causes naive routing between cheap and frontier models to increase costs, negating savings.

Outcome

The router achieved a $0.12 spend versus $1.23 frontier‑only on a sample run, saving 90% on that session and reducing overall frontier usage.

Workflow

• 1. Start a new conversation and send system prompt + first user message to router.
• 2. Router fingerprints the conversation and classifies difficulty (cheap, mid, frontier).
• 3. Router locks the conversation to the chosen tier.
• 4. Subsequent steps use the locked tier; if task becomes harder, a human can bump tier upward.
• 5. Downshifts only occur at cold boundaries (new conversation or reset).
• 6. Each request logs real cost and counterfactual frontier cost to a local ledger.
• 7. Review ledger for savings audit.

Evidence

coding-agent-router savings report: $0.12 spent vs $1.23 frontier-only, 90% saved on that run

Source

The design that actually works treats a conversation as the unit, not a request.