Confidence-Routed Agents: Cheap First, Frontier on Escalation

Описание и применение

Руководство по маршрутизации решений агента с использованием модели Jev для оценки уверенности и эскалации к Claude Sonnet 5 только при низкой уверенности, что снижает расходы вдвое и повышает точность.

Задача

How to balance cost and accuracy when routing uncertain agent decisions to a frontier model without overusing expensive LLMs.

Результат

Routing by Jev confidence and escalating only uncertain cases cut cost in half (from $0.0846 to $0.0422 for 240 calls) while raising accuracy from 91.7% to 93.3%. The gateway budget prevented runaway spending, and the router gracefully degraded when Jev was unreachable.

Рабочий процесс

• 1. Receive task from agent
• 2. Call Jev API to get answer and confidence
• 3. If confidence >= threshold, return Jev answer
• 4. If confidence < threshold, route to Claude Sonnet 5 via agentgateway
• 5. Enforce daily USD budget on escalation key
• 6. Return Claude answer

Доказательства

On a 240‑case benchmark it was also the worse one: routing by confidence and escalating only when a cheap model said it wasn’t sure cost half as much as escalating everything, and scored higher.

Источник

Routing at jev‑calibration's own recommended cutoff, escalate anything below exactly 1.000 confidence, cost half of what escalating everything costs ($0.0846 against an extrapolated $0.169 for 240 all‑Claude calls) and scored better, not worse: 93.3% against 91.7%.

Источник

agentgateway's own log line for the rejected call: 2026-10-01T14:39:20.199122Z warn budget API key budget exceeded ...

Источник