Homelab census: 41 containers, one 6 GB GPU, and where my AI agents run

Описание и применение

A homelab census of 41 Docker and 9 LXC containers, and how one 6 GB GPU decides which AI agents run on local Ollama, OpenRouter or a subscription.
Today I asked my LLM box what it was doing, and it told me it had a 27-billion-parameter model loaded. Total footprint 18.3 GB. Amount of that on the graphics card: 0.5 GB.

So the 27B was "running on the GPU" in the same sense that I am running a marathon when I walk to the shop. The other 17.8 GB sat in system RAM and did its arithmetic on the CPU, one patient token at a time.

That seemed like a good moment for a proper census: what is actually

Задача

A homelab census of 41 Docker and 9 LXC containers, and how one 6 GB GPU decides which AI agents run on local Ollama, OpenRouter or a subscription.
Today I asked my LLM box what it was doing, and it told me it had a 27-billion-parameter model loaded. Total footprint 18.3 GB. Amount of that on the graphics card: 0.5 GB.

So the 27B was "running on the GPU" in the same sense that I am running a marathon when I walk to the shop. The other 17.8 GB sat in system RAM and did its arithmetic on the CPU, one patient token at a time.

That seemed like a good moment for a proper census: what is actually

Результат

A homelab census of 41 Docker and 9 LXC containers, and how one 6 GB GPU decides which AI agents run on local Ollama, OpenRouter or a subscription.
Today I asked my LLM box what it was doing, and it told me it had a 27-billion-parameter model loaded. Total footprint 18.3 GB. Amount of that on the graphics card: 0.5 GB.

So the 27B was "running on the GPU" in the same sense that I am running a marathon when I walk to the shop. The other 17.8 GB sat in system RAM and did its arithmetic on the CPU, one patient token at a time.

That seemed like a good moment for a proper census: what is actually

Доказательства

Homelab census: 41 containers, one 6 GB GPU, and where my AI agents run

Источник