An AI optimization made llama2.c finish sooner and start streaming later

Description and use

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em></p> <p>I built a Kaggle benchmark to test whether coding agents can make a real CPU inference program faster while preserving its tested behavior. Gemini, Claude and GPT read, edit and test the C source of Karpathy’s llama2.c under the same tool-call and editing limits. Their resulting programs run TinyStories checkpoints of 15M, 42M and 110M parameters across several request workloads.</p> <p>The benchmark evaluates both the edited program and the effort needed to

Problem

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em></p> <p>I built a Kaggle benchmark to test whether coding agents can make a real CPU inference program faster while preserving its tested behavior. Gemini, Claude and GPT read, edit and test the C source of Karpathy’s llama2.c under the same tool-call and editing limits. Their resulting programs run TinyStories checkpoints of 15M, 42M and 110M parameters across several request workloads.</p> <p>The benchmark evaluates both the edited program and the effort needed to

Outcome

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em></p> <p>I built a Kaggle benchmark to test whether coding agents can make a real CPU inference program faster while preserving its tested behavior. Gemini, Claude and GPT read, edit and test the C source of Karpathy’s llama2.c under the same tool-call and editing limits. Their resulting programs run TinyStories checkpoints of 15M, 42M and 110M parameters across several request workloads.</p> <p>The benchmark evaluates both the edited program and the effort needed to

Evidence

An AI optimization made llama2.c finish sooner and start streaming later

Source