Can decision models like Jev and Laya route a coding agent?

Описание и применение

Модель Laya (421 М параметров) успешно интегрирована в прокси Torana для динамического выбора уровня модели (fast, balanced, strong, hold) при работе кода‑агента, добавляя ~112 мс задержки и позволяя экономить ресурсы на простых задачах.

Задача

Determine whether small decision models can effectively route a coding agent to appropriate language‑model tiers during a workflow.

Результат

Integrating Laya as a decision router reduced the need for a single, expensive model tier and allowed dynamic selection of cheaper models for routine tasks while reserving expensive models for complex reasoning. The router added only ~112 ms latency and was successfully tested with 48 synthetic coding prompts across four difficulty levels.

Рабочий процесс

• 1. Install Torana Edge 1fa71a2 with decision_router 0.6.1 and SDK 0.7.1. 2. Deploy Laya Python package 0.3.22 (English checkpoint, 421 M) locally on its System One endpoint. 3. Configure the decision_router plugin to watch a ladder of models (e.g., DeepSeek Flash → DeepSeek Pro). 4. Set the router mode to advise (or auto) and enable the optional classifier. 5. Run coding harnesses (Codex, Pi, Oh My Pi, OpenCode) through Torana with the router enabled. 6. For each user turn, the router sends the latest user message and local signals to Laya. 7. Laya returns a tier decision (fast, balanced, strong, hold) with probabilities and confidence. 8. Torana routes the request to the chosen model tier, respecting mode‑specific rules (shadow, advise, confirm, auto).

Доказательства

We plugged Laya, a 421M-parameter decision model, into our coding agents as a model router. The plumbing worked in about 112 ms.

Источник

Laya sees the latest user message and a few counters. It never sees code, tool output, the system prompt or earlier conversation.

Источник

The obvious first job was replacing the classifier a harness's auto mode uses to decide whether an action is safe.

Источник