One field in the request made our agent 3x cheaper and 8x faster

说明与用途

在LLM请求中指定推理努力级别，降低了Altair代理的成本和执行时间，提高了任务成功率。

问题

Reasoning models can spend excessive time and tokens when the level of effort is not specified, causing high cost and slow performance.

结果

Specifying a low reasoning effort cut reasoning tokens from 9,743 to 235 per hard task (41×), cost from 12.6 m$ to 4.2 m$ (3×), and time from 4.1 min to 0.55 min (7.5×). Adaptive mode achieved the same success rate as always‑high with lower cost and faster time.

工作流

• Run Altair on hard tasks with no reasoning level
• Measure cost, time, and success
• Add "reasoning":{"effort":"low"} to request
• Re‑run tasks and record metrics
• Test adaptive modes: always low, always high, high first step, low then high after failure
• Select best mode
• Update Altair code to set reasoning level per provider

证据

One field in the request body ("reasoning": {"effort": "low"}) made a task 3x cheaper and 7–8x faster

来源

Reasoning models decide by themselves how long to think if you don't tell them.

来源

12.6 m$ without a level, 4.2 m$ with "low": three times less.

来源

9,743 reasoning tokens per task on average without a level, 235 with "low" — 41x fewer.

来源

"think after a failure" solved as many as always‑high, at the cost and speed of low.

来源