I Stopped Paying for Colab Pro and Run LLMs on My Own GPU

Description and use

The author stopped paying for Colab Pro and ran all LLM inference and fine‑tuning locally on a RTX 3060, cutting monthly costs from $49.99 to $0 while keeping performance and data local.

Problem

High monthly cost of Colab Pro for running LLM inference and fine‑tuning on a consumer GPU.

Outcome

Monthly cost dropped from $49.99 to $0, with no disconnections or idle timeouts, and all data remains local.

Workflow

• 1. Install Ollama and pull desired model (e.g., qwen2.5:7b-instruct-q4_K_M).
• 2. Use Unsloth with the 3060 to fine‑tune models (e.g., 7B QLoRA).
• 3. Run inference locally via Ollama API or JupyterLab notebooks.
• 4. For occasional GPU memory overflow, use free Colab T4 tier.
• 5. Monitor power usage (~170 W) and adjust usage to keep electricity cost low.

Evidence

Total: $0/month vs $49.99/month. The only thing I still use Colab for is the occasional T4 session when I need CUDA memory I don't have — and the free tier covers that.

Source

Compute units burned: ~1,800/month (mostly idle timeouts I forgot to kill). Actual productive GPU hours: ~35/month. Effective cost per productive hour: $1.43.

Source