Our agent said "done" on 15% of tasks while the provider was failing
Описание и применение
Агент Altair ошибочно отмечал завершение задач из‑за ошибок провайдера, но после внедрения проверки ответов и повторных попыток процент ложных «завершений» снизился до нуля.
Задача
An AI agent reported task completion for 15% of tasks, but the provider returned errors or empty streams, causing false "done" signals.
Результат
False "done" rate dropped from 15% to near 0% after adding error detection and retry logic.
Рабочий процесс
- Run 46 tasks with Altair
- Log provider responses via proxy
- Detect false "done" using judge_turn
- Retry or report error
- Re‑run tasks with updated logic
Доказательства
7 weren't, and in all seven the provider broke: 6 times — the gateway's error text instead of an answer. Once — an empty stream.
We now check every answer without a tool call. It's not an answer if: ...
- Дата записи: 2026-10-05T13:56:01.606Z