Running a 180B Model on a Gaming Laptop: POCKET-Darwin-180B

Description and use

A 180B model runs on a gaming laptop with 8 GB VRAM using sparse MoE, SSD streaming, and llama.cpp, enabling offline operation in air‑gapped environments.

Problem

Deploying a 180‑parameter language model on a consumer gaming laptop with limited VRAM and system memory, while maintaining offline, air‑gapped operation.

Outcome

The model operates offline on a standard gaming laptop, demonstrating that 180B‑parameter inference is feasible without cloud infrastructure.

Workflow

• 1. Quantize the 180B model weights for llama.cpp.
• 2. Store expert weights on SSD and configure SSD streaming.
• 3. Load the model into llama.cpp runtime on the laptop.
• 4. Run inference; only the active experts are streamed into VRAM and RAM per token.

Evidence

VIDRAFT released POCKET-Darwin-180B, a 180B-parameter model that runs on a consumer gaming laptop with 8GB of VRAM and 32GB of system memory.

Source

Three ideas work together. Sparse Mixture of Experts, SSD streaming, and llama.cpp runtime.

Source

The reported setup is a gaming laptop with 8GB of VRAM and 32GB of system memory, with weights streamed from SSD.

Source