I made AI coding agents race each other, and the fast one wasn't the one I expected

Description and use

The author built Agent Derby to race AI coding agents side‑by‑side on the same prompt. The fastest agent finished in 17 s, but the most feature‑rich agent took 2½ min. The experiment showed speed does not equal overall quality.

Problem

Choosing the best AI coding agent based on benchmarks that may not reflect real-world performance.

Outcome

The fastest agent (Sonnet 5.5) completed the snake game in 17 s, but the most feature‑rich and cost‑effective agent (Opus 5.5) produced a richer game in 2½ min. The race revealed that speed does not equal overall quality.

Workflow

• 1. Type a task and select agents to race.
• 2. Each agent receives the same prompt simultaneously.
• 3. Agents run in isolated sandboxes, preventing interference.
• 4. Real-time view shows each agent’s file edits, terminal output, clock, token usage, and cost.
• 5. When an agent finishes, its built artifact launches automatically (web app in frame, terminal app in terminal).
• 6. Results are displayed in a comparison table with ranking, cost, and token metrics.
• 7. Optionally keep a result as a new branch or folder; no automatic merging.

Evidence

Every few days someone tells me which coding agent is "obviously the best".

Source

I typed "build a playable snake game in the browser", picked three Claude models, and hit start.

Source

Sonnet 5.5 was finished in 17 seconds.

Source

Opus built a start screen, sound effects, a pause button and a mute key. It wrote about six times as much code.

Source

If an agent's CLI doesn't report a number, the app says "not reported".

Source