Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

Running a 180B Model on a Gaming Laptop: POCKET-Darwin-180B

Описание и применение

Модель 180B запущена на игровой ноутбуке с 8 ГБ видеопамяти, используя sparse MoE, SSD‑streaming и llama.cpp, что позволяет работать офлайн в изолированных средах.

Задача

Deploying a 180‑parameter language model on a consumer gaming laptop with limited VRAM and system memory, while maintaining offline, air‑gapped operation.

Результат

The model operates offline on a standard gaming laptop, demonstrating that 180B‑parameter inference is feasible without cloud infrastructure.

Рабочий процесс

  • 1. Quantize the 180B model weights for llama.cpp.
  • 2. Store expert weights on SSD and configure SSD streaming.
  • 3. Load the model into llama.cpp runtime on the laptop.
  • 4. Run inference; only the active experts are streamed into VRAM and RAM per token.

Доказательства

VIDRAFT released POCKET-Darwin-180B, a 180B-parameter model that runs on a consumer gaming laptop with 8GB of VRAM and 32GB of system memory.

Источник

Three ideas work together. Sparse Mixture of Experts, SSD streaming, and llama.cpp runtime.

Источник

The reported setup is a gaming laptop with 8GB of VRAM and 32GB of system memory, with weights streamed from SSD.

Источник

  • Дата записи: 2026-10-08T02:40:48.230Z