40 Lines of Go That Cut Our LLM Bill by 71%

说明与用途

40 行 Go 代码将 81% 的请求路由到廉价 GPT‑5.6 Luna 模型，降低 LLM 成本 71%。

问题

High cost of using expensive LLMs for routine, low‑complexity requests.

结果

The 40‑line Go router moved 81% of traffic to the cheap model and cut the LLM bill by 71% while maintaining quality for complex requests.

工作流

• 1. Send request to cheap model (Luna).
• 2. Inspect the output and finish reason.
• 3. If any gate fails, re‑run the request on the strong model.
• 4. Return the strong model’s output; otherwise return the cheap model’s output.

证据

It moved 81% of requests off the expensive model, and it cut the bill by 71%.

来源

The rule: don’t classify the prompt. Run the cheap model and judge the output.

来源