Supercharge Your LLM Router with a Decision Model: Clef on Cloudflare Workers

Description and use

By swapping the verbose Llama 4 Scout classifier for Cloudflare’s Clef decision model, the author raised routing accuracy from 75 % to 85 %, eliminated unnecessary text generation, and reduced costs.

Problem

The existing LLM router misclassified prompts, routing coding questions to a cheap model and general prompts to a costly model, leading to wasted costs and incorrect responses.

Outcome

Replacing the verbose LLM classifier with a decision model eliminated unnecessary text generation, improved routing accuracy from 75% to 85%, and reduced costs by ensuring coding prompts reach the expensive model and general prompts reach the cheap model.

Workflow

• 1. Add a /classify endpoint that runs only the classifier and returns JSON.
• 2. Create a benchmark script (scripts/bench.mjs) to send 20 labelled prompts through /classify multiple times and report accuracy.
• 3. Compare Llama 4 Scout, Clef, and Clef‑flash by setting the x‑classifier header per request.
• 4. Observe that Llama misclassifies some coding prompts and confidently misclassifies three general prompts.
• 5. Implement a safer default that routes only exact "simple" to the cheap model, others to the strong model, raising accuracy to 85%.
• 6. Replace the classifier registry entry with Clef, defining a choice question with options "coding" and "simple" and criteria descriptions.
• 7. Deploy the updated router and benchmark again to confirm improved accuracy and reduced text generation.

Evidence

The classifier was a small LLM, Llama 4 Scout, with a one‑line system prompt: "Reply with only that single word."

Source

The first run of Llama 4 Scout scored 75% accuracy… It ignored "one word"…

Source

Clef is a decision model: you give it the content and typed questions, and it returns typed answers. There is no generated text, so there is nothing to ramble and nothing to parse.

Source