Our in‑house agent router: Jev vs Kev-4B vs bge-small

Description and use

An in‑house agent router uses two local models, bge‑small and Kev‑4B, to route prompts to the correct agent with 99.5 % detection accuracy and 85 % correct agent selection, zero cost per call, and no data leaving the premises.

Problem

Routing prompts to the correct business automation agent while keeping data on premises and minimizing cost.

Outcome

Using bge‑small and Kev‑4B achieves ~99.5% detection, 8% false alarm, 85% correct agent selection (97% in top 3) with ~0.12 s latency and zero cost per call.

Workflow

• Receive prompt
• Run bge‑small (6 ms) to detect off‑goal prompts
• Run Kev‑4B (~0.12 s) to detect off‑goal prompts
• If both models reject, Kev‑4B selects top 3 candidate agents
• Route prompt to selected agent
• If both models accept, keep prompt with current agent

Evidence

bge‑small (6 ms) is very accurate at catching prompts that don't belong. But it is not good at picking the right agent (in our experiments).

Source

Kev‑4B (local, ~0.12 s) is slightly lower at calling out prompts that don't belong, but good at picking the right target agent: it is in its top 3 for 97% of prompts, against 94% for bge‑small.

Source

Detection: an off‑goal prompt is flagged. False alarm: an on‑goal prompt is wrongly flagged. Right agent for the prompt: an other‑goal prompt is sent to the correct agent.

Source