Run DeepSeek-V4.1-Flash (748B) locally with Project Maya on one RTX 5090

说明与用途

DeepSeek‑V4.1‑Flash（748 B）在单个 RTX 5090 上使用 Project Maya 本地运行，下载约350 GB模型文件后，在262 K上下文下实现约21 tokens/s。

问题

Run the 748B DeepSeek-V4.1-Flash model locally on a single RTX 5090 GPU with limited RAM (128 GB) and NVMe storage.

结果

DeepSeek-V4.1-Flash runs locally on one RTX 5090, achieving ~21 tokens/s with 262 K context, after downloading ~350 GB of model files and configuring Maya‑DS4.

工作流

• Clone Project Maya repository
• Run maya.sh --check to install dependencies
• Run maya.sh --setup with flags: --model DeepSeek-V4.1-Flash-1017 --ds4-quant maya-s --context 262144 --vram-reserve-mib 3072 --no-vision --port 8082 --no-start
• Start the server in a systemd slice without memory limits using systemd-run
• Send a chat completion request via curl to http://127.0.0.1:8082/v1/chat/completions

证据

DeepSeek-V4.1-Flash-1017 has 748B parameters. Even Project Maya's own 2-3 bit quant is 340 GB with its Engram tables, and my workstation has 128 GB of RAM.

来源

The first start after setup also runs Maya's speed calibration (below) and took about four minutes here; a normal start took 20 seconds.

来源

The flags that matter: --model DeepSeek-V4.1-Flash-1017 --ds4-quant maya-s picks the model and the smallest quant.

来源