Can decision models like Jev and Laya route a coding agent?

说明与用途

Laya（421 M 参数）被集成到 Torana 中，以动态路由编码代理请求到适当的模型层级，增加约 112 ms 延迟，并在简单任务上实现成本节约。

问题

Determine whether small decision models can effectively route a coding agent to appropriate language‑model tiers during a workflow.

结果

Integrating Laya as a decision router reduced the need for a single, expensive model tier and allowed dynamic selection of cheaper models for routine tasks while reserving expensive models for complex reasoning. The router added only ~112 ms latency and was successfully tested with 48 synthetic coding prompts across four difficulty levels.

工作流

• 1. Install Torana Edge 1fa71a2 with decision_router 0.6.1 and SDK 0.7.1. 2. Deploy Laya Python package 0.3.22 (English checkpoint, 421 M) locally on its System One endpoint. 3. Configure the decision_router plugin to watch a ladder of models (e.g., DeepSeek Flash → DeepSeek Pro). 4. Set the router mode to advise (or auto) and enable the optional classifier. 5. Run coding harnesses (Codex, Pi, Oh My Pi, OpenCode) through Torana with the router enabled. 6. For each user turn, the router sends the latest user message and local signals to Laya. 7. Laya returns a tier decision (fast, balanced, strong, hold) with probabilities and confidence. 8. Torana routes the request to the chosen model tier, respecting mode‑specific rules (shadow, advise, confirm, auto).

证据

We plugged Laya, a 421M-parameter decision model, into our coding agents as a model router. The plumbing worked in about 112 ms.

来源

Laya sees the latest user message and a few counters. It never sees code, tool output, the system prompt or earlier conversation.

来源

The obvious first job was replacing the classifier a harness's auto mode uses to decide whether an action is safe.

来源