Can decision models like Jev and Laya route a coding agent?

Description and use

Laya (421 M parameters) was integrated into Torana to dynamically route coding agent requests to appropriate model tiers, adding ~112 ms latency and enabling cost savings on simple tasks.

Problem

Determine whether small decision models can effectively route a coding agent to appropriate language‑model tiers during a workflow.

Outcome

Integrating Laya as a decision router reduced the need for a single, expensive model tier and allowed dynamic selection of cheaper models for routine tasks while reserving expensive models for complex reasoning. The router added only ~112 ms latency and was successfully tested with 48 synthetic coding prompts across four difficulty levels.

Workflow

• 1. Install Torana Edge 1fa71a2 with decision_router 0.6.1 and SDK 0.7.1. 2. Deploy Laya Python package 0.3.22 (English checkpoint, 421 M) locally on its System One endpoint. 3. Configure the decision_router plugin to watch a ladder of models (e.g., DeepSeek Flash → DeepSeek Pro). 4. Set the router mode to advise (or auto) and enable the optional classifier. 5. Run coding harnesses (Codex, Pi, Oh My Pi, OpenCode) through Torana with the router enabled. 6. For each user turn, the router sends the latest user message and local signals to Laya. 7. Laya returns a tier decision (fast, balanced, strong, hold) with probabilities and confidence. 8. Torana routes the request to the chosen model tier, respecting mode‑specific rules (shadow, advise, confirm, auto).

Evidence

We plugged Laya, a 421M-parameter decision model, into our coding agents as a model router. The plumbing worked in about 112 ms.

Source

Laya sees the latest user message and a few counters. It never sees code, tool output, the system prompt or earlier conversation.

Source

The obvious first job was replacing the classifier a harness's auto mode uses to decide whether an action is safe.

Source