One field in the request made our agent 3x cheaper and 8x faster

Описание и применение

Указание уровня усилий в запросе к модели LLM сократило расходы и время выполнения задач агента Altair, повысив эффективность и точность.

Задача

Reasoning models can spend excessive time and tokens when the level of effort is not specified, causing high cost and slow performance.

Результат

Specifying a low reasoning effort cut reasoning tokens from 9,743 to 235 per hard task (41×), cost from 12.6 m$ to 4.2 m$ (3×), and time from 4.1 min to 0.55 min (7.5×). Adaptive mode achieved the same success rate as always‑high with lower cost and faster time.

Рабочий процесс

• Run Altair on hard tasks with no reasoning level
• Measure cost, time, and success
• Add "reasoning":{"effort":"low"} to request
• Re‑run tasks and record metrics
• Test adaptive modes: always low, always high, high first step, low then high after failure
• Select best mode
• Update Altair code to set reasoning level per provider

Доказательства

One field in the request body ("reasoning": {"effort": "low"}) made a task 3x cheaper and 7–8x faster

Источник

Reasoning models decide by themselves how long to think if you don't tell them.

Источник

12.6 m$ without a level, 4.2 m$ with "low": three times less.

Источник

9,743 reasoning tokens per task on average without a level, 235 with "low" — 41x fewer.

Источник

"think after a failure" solved as many as always‑high, at the cost and speed of low.

Источник