Running a 180B Model on a Gaming Laptop: POCKET-Darwin-180B
Описание и применение
Модель 180B запущена на игровой ноутбуке с 8 ГБ видеопамяти, используя sparse MoE, SSD‑streaming и llama.cpp, что позволяет работать офлайн в изолированных средах.
Задача
Deploying a 180‑parameter language model on a consumer gaming laptop with limited VRAM and system memory, while maintaining offline, air‑gapped operation.
Результат
The model operates offline on a standard gaming laptop, demonstrating that 180B‑parameter inference is feasible without cloud infrastructure.
Рабочий процесс
- 1. Quantize the 180B model weights for llama.cpp.
- 2. Store expert weights on SSD and configure SSD streaming.
- 3. Load the model into llama.cpp runtime on the laptop.
- 4. Run inference; only the active experts are streamed into VRAM and RAM per token.
Доказательства
VIDRAFT released POCKET-Darwin-180B, a 180B-parameter model that runs on a consumer gaming laptop with 8GB of VRAM and 32GB of system memory.
Three ideas work together. Sparse Mixture of Experts, SSD streaming, and llama.cpp runtime.
The reported setup is a gaming laptop with 8GB of VRAM and 32GB of system memory, with weights streamed from SSD.
- Дата записи: 2026-10-08T02:40:48.230Z