40 Lines of Go That Cut Our LLM Bill by 71%

Описание и применение

40‑строчный Go‑код, который перенаправляет 81 % запросов к дешёвой модели GPT‑5.6 Luna, снижая расходы на LLM на 71 %.

Задача

High cost of using expensive LLMs for routine, low‑complexity requests.

Результат

The 40‑line Go router moved 81% of traffic to the cheap model and cut the LLM bill by 71% while maintaining quality for complex requests.

Рабочий процесс

• 1. Send request to cheap model (Luna).
• 2. Inspect the output and finish reason.
• 3. If any gate fails, re‑run the request on the strong model.
• 4. Return the strong model’s output; otherwise return the cheap model’s output.

Доказательства

It moved 81% of requests off the expensive model, and it cut the bill by 71%.

Источник

The rule: don’t classify the prompt. Run the cheap model and judge the output.

Источник