What broke when we ran agents on small local models

Description and use

<p>Bees runs a team of AI agents on your own computer. You can point it at Claude Code, Codex or an API key, but we also ship a local model so you can run it without paying for an API. That model runs through llama.cpp on a laptop, and it is small: 3 to 4 billion parameters.</p> <p>Getting agents to do real work on a model that size took us most of August and September. Almost none of the failures were "the model is too dumb". Most of them were our code, or a mismatch between our code and the model, and they looked exactly like a dumb model from the outside.</p> <p>Here are the five that cost

Problem

<p>Bees runs a team of AI agents on your own computer. You can point it at Claude Code, Codex or an API key, but we also ship a local model so you can run it without paying for an API. That model runs through llama.cpp on a laptop, and it is small: 3 to 4 billion parameters.</p> <p>Getting agents to do real work on a model that size took us most of August and September. Almost none of the failures were "the model is too dumb". Most of them were our code, or a mismatch between our code and the model, and they looked exactly like a dumb model from the outside.</p> <p>Here are the five that cost

Outcome

<p>Bees runs a team of AI agents on your own computer. You can point it at Claude Code, Codex or an API key, but we also ship a local model so you can run it without paying for an API. That model runs through llama.cpp on a laptop, and it is small: 3 to 4 billion parameters.</p> <p>Getting agents to do real work on a model that size took us most of August and September. Almost none of the failures were "the model is too dumb". Most of them were our code, or a mismatch between our code and the model, and they looked exactly like a dumb model from the outside.</p> <p>Here are the five that cost

Evidence

What broke when we ran agents on small local models

Source