40 Lines of Go That Cut Our LLM Bill by 71%

Description and use

A 40‑line Go router that sends 81 % of requests to the cheap GPT‑5.6 Luna model, cutting LLM costs by 71 %.

Problem

High cost of using expensive LLMs for routine, low‑complexity requests.

Outcome

The 40‑line Go router moved 81% of traffic to the cheap model and cut the LLM bill by 71% while maintaining quality for complex requests.

Workflow

• 1. Send request to cheap model (Luna).
• 2. Inspect the output and finish reason.
• 3. If any gate fails, re‑run the request on the strong model.
• 4. Return the strong model’s output; otherwise return the cheap model’s output.

Evidence

It moved 81% of requests off the expensive model, and it cut the bill by 71%.

Source

The rule: don’t classify the prompt. Run the cheap model and judge the output.

Source