I Built an AI Agent That Optimizes Code — But Benchmarks Decide What Survives

Описание и применение

Автономный агент генерирует и тестирует оптимизации кода, используя замороженные бенчмарки и тесты для объективного выбора улучшений.

Задача

AI coding agents can propose optimizations that look good but may not actually improve performance or correctness.

Результат

The system automates performance optimization by letting an AI agent iterate through changes, with a benchmark and test suite acting as an external judge to ensure only genuinely faster and correct code is retained.

Рабочий процесс

• Agent proposes an optimization
• Agent applies the change
• Run unit tests
• Run frozen benchmark
• Measure performance
• If performance improves and tests pass, KEEP the change; otherwise DISCARD
• Repeat

Доказательства

The agent does not decide whether its own change is good. Instead, every experiment is evaluated by a frozen benchmark and test harness.

Источник

If the change improves the measured result and still passes the required checks, it can survive.

Источник