Supercharge Your LLM Router with a Decision Model: Clef on Cloudflare Workers

说明与用途

将冗长的 Llama 4 Scout 分类器替换为 Cloudflare 的 Clef 决策模型后，作者将路由准确率从 75% 提升至 85%，消除了不必要的文本生成并降低了成本。

问题

The existing LLM router misclassified prompts, routing coding questions to a cheap model and general prompts to a costly model, leading to wasted costs and incorrect responses.

结果

Replacing the verbose LLM classifier with a decision model eliminated unnecessary text generation, improved routing accuracy from 75% to 85%, and reduced costs by ensuring coding prompts reach the expensive model and general prompts reach the cheap model.

工作流

• 1. Add a /classify endpoint that runs only the classifier and returns JSON.
• 2. Create a benchmark script (scripts/bench.mjs) to send 20 labelled prompts through /classify multiple times and report accuracy.
• 3. Compare Llama 4 Scout, Clef, and Clef‑flash by setting the x‑classifier header per request.
• 4. Observe that Llama misclassifies some coding prompts and confidently misclassifies three general prompts.
• 5. Implement a safer default that routes only exact "simple" to the cheap model, others to the strong model, raising accuracy to 85%.
• 6. Replace the classifier registry entry with Clef, defining a choice question with options "coding" and "simple" and criteria descriptions.
• 7. Deploy the updated router and benchmark again to confirm improved accuracy and reduced text generation.

证据

The classifier was a small LLM, Llama 4 Scout, with a one‑line system prompt: "Reply with only that single word."

来源

The first run of Llama 4 Scout scored 75% accuracy… It ignored "one word"…

来源

Clef is a decision model: you give it the content and typed questions, and it returns typed answers. There is no generated text, so there is nothing to ramble and nothing to parse.

来源