Running a 180B Model on a Gaming Laptop: POCKET-Darwin-180B

说明与用途

180B 模型在配备8GB显存的游戏笔记本上运行，采用稀疏MoE、SSD流式和llama.cpp，实现离线、隔离环境下的部署。

问题

Deploying a 180‑parameter language model on a consumer gaming laptop with limited VRAM and system memory, while maintaining offline, air‑gapped operation.

结果

The model operates offline on a standard gaming laptop, demonstrating that 180B‑parameter inference is feasible without cloud infrastructure.

工作流

• 1. Quantize the 180B model weights for llama.cpp.
• 2. Store expert weights on SSD and configure SSD streaming.
• 3. Load the model into llama.cpp runtime on the laptop.
• 4. Run inference; only the active experts are streamed into VRAM and RAM per token.

证据

VIDRAFT released POCKET-Darwin-180B, a 180B-parameter model that runs on a consumer gaming laptop with 8GB of VRAM and 32GB of system memory.

来源

Three ideas work together. Sparse Mixture of Experts, SSD streaming, and llama.cpp runtime.

来源

The reported setup is a gaming laptop with 8GB of VRAM and 32GB of system memory, with weights streamed from SSD.

来源