I built a router to cut my Claude Code bill, and prompt caching was the whole problem

说明与用途

一个路由器按会话锁定模型层级，避免缓存失效，节省高达90%的Claude Code成本。

问题

Prompt caching causes naive routing between cheap and frontier models to increase costs, negating savings.

结果

The router achieved a $0.12 spend versus $1.23 frontier‑only on a sample run, saving 90% on that session and reducing overall frontier usage.

工作流

• 1. Start a new conversation and send system prompt + first user message to router.
• 2. Router fingerprints the conversation and classifies difficulty (cheap, mid, frontier).
• 3. Router locks the conversation to the chosen tier.
• 4. Subsequent steps use the locked tier; if task becomes harder, a human can bump tier upward.
• 5. Downshifts only occur at cold boundaries (new conversation or reset).
• 6. Each request logs real cost and counterfactual frontier cost to a local ledger.
• 7. Review ledger for savings audit.

证据

coding-agent-router savings report: $0.12 spent vs $1.23 frontier-only, 90% saved on that run

来源

The design that actually works treats a conversation as the unit, not a request.