Supercharge Your LLM Router with a Decision Model: Clef on Cloudflare Workers

Описание и применение

Заменив громоздкий классификатор Llama 4 Scout на решение Clef, автор улучшил точность маршрутизации запросов от 75 % до 85 %, устранив лишнюю генерацию текста и экономя расходы.

Задача

The existing LLM router misclassified prompts, routing coding questions to a cheap model and general prompts to a costly model, leading to wasted costs and incorrect responses.

Результат

Replacing the verbose LLM classifier with a decision model eliminated unnecessary text generation, improved routing accuracy from 75% to 85%, and reduced costs by ensuring coding prompts reach the expensive model and general prompts reach the cheap model.

Рабочий процесс

• 1. Add a /classify endpoint that runs only the classifier and returns JSON.
• 2. Create a benchmark script (scripts/bench.mjs) to send 20 labelled prompts through /classify multiple times and report accuracy.
• 3. Compare Llama 4 Scout, Clef, and Clef‑flash by setting the x‑classifier header per request.
• 4. Observe that Llama misclassifies some coding prompts and confidently misclassifies three general prompts.
• 5. Implement a safer default that routes only exact "simple" to the cheap model, others to the strong model, raising accuracy to 85%.
• 6. Replace the classifier registry entry with Clef, defining a choice question with options "coding" and "simple" and criteria descriptions.
• 7. Deploy the updated router and benchmark again to confirm improved accuracy and reduced text generation.

Доказательства

The classifier was a small LLM, Llama 4 Scout, with a one‑line system prompt: "Reply with only that single word."

Источник

The first run of Llama 4 Scout scored 75% accuracy… It ignored "one word"…

Источник

Clef is a decision model: you give it the content and typed questions, and it returns typed answers. There is no generated text, so there is nothing to ramble and nothing to parse.

Источник