I Stopped Paying for Colab Pro and Run LLMs on My Own GPU

说明与用途

作者停止使用 Colab Pro，将所有 LLM 推理和微调迁移到本地 RTX 3060，月费用从 $49.99 降至 $0，同时保持性能和数据本地化。

问题

High monthly cost of Colab Pro for running LLM inference and fine‑tuning on a consumer GPU.

结果

Monthly cost dropped from $49.99 to $0, with no disconnections or idle timeouts, and all data remains local.

工作流

• 1. Install Ollama and pull desired model (e.g., qwen2.5:7b-instruct-q4_K_M).
• 2. Use Unsloth with the 3060 to fine‑tune models (e.g., 7B QLoRA).
• 3. Run inference locally via Ollama API or JupyterLab notebooks.
• 4. For occasional GPU memory overflow, use free Colab T4 tier.
• 5. Monitor power usage (~170 W) and adjust usage to keep electricity cost low.

证据

Total: $0/month vs $49.99/month. The only thing I still use Colab for is the occasional T4 session when I need CUDA memory I don't have — and the free tier covers that.

来源

Compute units burned: ~1,800/month (mostly idle timeouts I forgot to kill). Actual productive GPU hours: ~35/month. Effective cost per productive hour: $1.43.

来源