Confidence-Routed Agents: Cheap First, Frontier on Escalation

Description and use

Guide on routing agent decisions using Jev for confidence scoring and escalating to Claude Sonnet 5 only when confidence is low, cutting costs by half and improving accuracy.

Problem

How to balance cost and accuracy when routing uncertain agent decisions to a frontier model without overusing expensive LLMs.

Outcome

Routing by Jev confidence and escalating only uncertain cases cut cost in half (from $0.0846 to $0.0422 for 240 calls) while raising accuracy from 91.7% to 93.3%. The gateway budget prevented runaway spending, and the router gracefully degraded when Jev was unreachable.

Workflow

• 1. Receive task from agent
• 2. Call Jev API to get answer and confidence
• 3. If confidence >= threshold, return Jev answer
• 4. If confidence < threshold, route to Claude Sonnet 5 via agentgateway
• 5. Enforce daily USD budget on escalation key
• 6. Return Claude answer

Evidence

On a 240‑case benchmark it was also the worse one: routing by confidence and escalating only when a cheap model said it wasn’t sure cost half as much as escalating everything, and scored higher.

Source

Routing at jev‑calibration's own recommended cutoff, escalate anything below exactly 1.000 confidence, cost half of what escalating everything costs ($0.0846 against an extrapolated $0.169 for 240 all‑Claude calls) and scored better, not worse: 93.3% against 91.7%.

Source

agentgateway's own log line for the rejected call: 2026-10-01T14:39:20.199122Z warn budget API key budget exceeded ...

Source