Homelab census: 41 containers, one 6 GB GPU, and where my AI agents run

说明与用途

作者在家庭实验室中对容器和 GPU 内存进行了详细清点，制定了基于上下文窗口和输入成本的本地与云模型路由规则，并实现了监控和故障转移机制。

问题

The author needed to understand what AI workloads were running on which hardware, how GPU memory was being used, and how to route tasks between local and cloud models to balance cost, latency, and security.

结果

The author achieved a clear mapping of AI workloads to hardware, identified GPU memory constraints, and established a cost‑aware routing strategy that balances local privacy with cloud speed. The system now automatically fails over to local or cloud models based on health checks and credit limits.

工作流

• 1. Inventory all physical nodes and their hardware specs.
• 2. List all LXC and Docker containers on each node and their purposes.
• 3. Measure GPU memory usage for each loaded model (e.g., 27B uses 0.5 GB on GPU, 17.8 GB on CPU).
• 4. Calculate total CPU threads and RAM usage across all nodes.
• 5. Identify duplicate or stale containers (e.g., twin agent box).
• 6. Define routing rule: use local model only if it meets 64K context and fits GPU; otherwise use cloud.
• 7. Assign each job to a target (local Ollama, OpenRouter, OpenCode, ComfyUI, Whisper/Piper).
• 8. Implement watchdogs to monitor model health and credit balance, with failover logic.
• 9. Measure cost and performance of cloud models over 30 days to inform pricing decisions.
• 10. Optimize local workloads by unloading idle models and reducing prompt size.
• 11. Document findings and share lessons learned.

证据

The 27B was "running on the GPU" in the same sense that I am running a marathon when I walk to the shop.

来源

The rule the whole routing falls out of: the agent framework I run refuses any model with less than a 64K context window

来源

On 7 September every paid model "failed" its health check at once, so the watchdog moved four agents back to the local model.

来源

The agent moved there for the extra RAM, and it was the right call, but every reboot is a queue of cold‑starting databases fighting over one spindle.

来源