Confidence-Routed Agents: Cheap First, Frontier on Escalation

说明与用途

使用Jev评估置信度并仅在置信度低时将决策升级到Claude Sonnet 5的代理路由指南，成本降低一半，准确率提升。

问题

How to balance cost and accuracy when routing uncertain agent decisions to a frontier model without overusing expensive LLMs.

结果

Routing by Jev confidence and escalating only uncertain cases cut cost in half (from $0.0846 to $0.0422 for 240 calls) while raising accuracy from 91.7% to 93.3%. The gateway budget prevented runaway spending, and the router gracefully degraded when Jev was unreachable.

工作流

• 1. Receive task from agent
• 2. Call Jev API to get answer and confidence
• 3. If confidence >= threshold, return Jev answer
• 4. If confidence < threshold, route to Claude Sonnet 5 via agentgateway
• 5. Enforce daily USD budget on escalation key
• 6. Return Claude answer

证据

On a 240‑case benchmark it was also the worse one: routing by confidence and escalating only when a cheap model said it wasn’t sure cost half as much as escalating everything, and scored higher.

来源

Routing at jev‑calibration's own recommended cutoff, escalate anything below exactly 1.000 confidence, cost half of what escalating everything costs ($0.0846 against an extrapolated $0.169 for 240 all‑Claude calls) and scored better, not worse: 93.3% against 91.7%.

来源

agentgateway's own log line for the rejected call: 2026-10-01T14:39:20.199122Z warn budget API key budget exceeded ...

来源