I made AI coding agents race each other, and the fast one wasn't the one I expected

Описание и применение

Разработчик создал инструмент Agent Derby, позволяющий сравнивать AI‑кодировщиков в реальном времени на одной задаче. Быстрый агент завершил задачу за 17 с, но более дорогой агент создал более функциональное приложение за 2½ мин. Это показало, что скорость не всегда означает лучшую общую производительность.

Задача

Choosing the best AI coding agent based on benchmarks that may not reflect real-world performance.

Результат

The fastest agent (Sonnet 5.5) completed the snake game in 17 s, but the most feature‑rich and cost‑effective agent (Opus 5.5) produced a richer game in 2½ min. The race revealed that speed does not equal overall quality.

Рабочий процесс

• 1. Type a task and select agents to race.
• 2. Each agent receives the same prompt simultaneously.
• 3. Agents run in isolated sandboxes, preventing interference.
• 4. Real-time view shows each agent’s file edits, terminal output, clock, token usage, and cost.
• 5. When an agent finishes, its built artifact launches automatically (web app in frame, terminal app in terminal).
• 6. Results are displayed in a comparison table with ranking, cost, and token metrics.
• 7. Optionally keep a result as a new branch or folder; no automatic merging.

Доказательства

Every few days someone tells me which coding agent is "obviously the best".

Источник

I typed "build a playable snake game in the browser", picked three Claude models, and hit start.

Источник

Sonnet 5.5 was finished in 17 seconds.

Источник

Opus built a start screen, sound effects, a pause button and a mute key. It wrote about six times as much code.

Источник

If an agent's CLI doesn't report a number, the app says "not reported".

Источник