Why Make an LLM Do the Grunt Work? After Jev, Who Should Handle an Agent’s Tool Work?

说明与用途

通过专门的子模型 Xiaoheigong 自动化 LLM 代理的工具交互，降低令牌使用和成本，让前沿模型专注于高层意图。

问题

Freelance writers and developers using LLM agents for long‑running tasks spend excessive compute and time on trivial tool‑selection and argument‑preparation steps, inflating costs and slowing productivity.

结果

By delegating all tool‑related work to a specialized sub‑model and organizing tools into capability groups, the author reduced token usage, cut costs, and improved task completion speed while keeping the frontier model focused on high‑level intent.

工作流

• 1. Expose a small set of common tools and use search for others.
• 2. Dynamically recommend tool sets at task boundaries, keeping them stable within a work unit.
• 3. Group tools into capability sets and expand the full set for each group.
• 4. Provide runtime.find and runtime.describe for on‑demand tool discovery and documentation.
• 5. Package deterministic procedures into reusable Operators to reduce repeated program generation.
• 6. Train a specialized sub‑model (Xiaoheigong) to handle tool discovery, argument preparation, execution, error handling, and result verification.
• 7. Escalate only when the task scope or intent is ambiguous or incomplete.

证据

I built more than 600 tools. They were useful. Very useful. But the tool descriptions kept growing, the bills kept growing, and tasks kept getting slower.

来源

I started trying to reduce the cost.

来源