A 150B MoE on a Developer Laptop: An Upper Bound on What Faster Storage Buys You

说明与用途

在笔记本上，150B MoE 模型运行时间为180秒，磁盘仅占10.7秒。更快的NVMe仅提升约6%，因此内存仍然至关重要。

问题

Can a developer run a 150B mixture‑of‑experts model on a typical engineering laptop, and how much does storage speed actually affect performance?

结果

On a 2026 engineering laptop, storage contributes only ~6 % of total latency for a 150B MoE model; faster NVMe drives yield marginal gains. Memory allocation for hot experts remains critical, as reducing RAM cache significantly degrades throughput.

工作流

• 1. Install Colibrì v1.11.0 with gcc 13.3.0 and O3 -march=native flags. 2. Download DeepSeek V4 Flash 150B REAP‑pruned checkpoint. 3. Run Colibrì’s iobench to measure NVMe bandwidth (O_DIRECT, 13 MB blocks). 4. Configure Colibrì planner: set RAM budget, enable read/compute overlap, disable speculative decoding. 5. Execute a 200‑token generation with coli run and record timing, hit/miss stats. 6. Calculate physical I/O bytes from v4_direct reads + fallbacks. 7. Compute device time: bytes ÷ measured bandwidth. 8. Repeat with halved RAM cache (20 GB) after dropping page cache to observe impact on misses and throughput.

证据

Two hundred tokens in 180.1 seconds: 1.11 tokens per second, or 0.90 seconds per token.

60.6 GB delivered by a drive measured at 5.67 GB/s is 10.7 seconds of device time across a 180-second generation.

Halving the cache cost 15% of throughput, so misses are not free and the engine's own advice to give it memory is right.