DFlash-2: Benchmarking Z-Lab's Successor to DFlash for Accuracy and Throughput Gains

说明与用途

DFlash‑2 在 Muse Glimmer 30B 上提升了吞吐量和草稿令牌排序准确性，使用 RTX A4000 和 llama.cpp 进行验证。

问题

Determine whether DFlash-2 improves throughput and accuracy over DFlash and a baseline model on Muse Glimmer 30B.

结果

DFlash‑2 achieved higher token throughput than DFlash while reducing the drop‑off in draft acceptance toward the end of a block, demonstrating that the lightweight path selector and local convolution effectively improve draft‑token ordering accuracy.

工作流

• Clone llama.cpp and apply PR #27342 for DFlash‑2 support
• Build llama.cpp with CUDA enabled
• Download Muse‑Glimmer‑30B‑UD‑Q2_K_XL.gguf, Muse‑Glimmer‑30B‑DFlash2‑Q4_K_M.gguf, and dflash‑kquant.gguf
• Start llama‑server with --model, --model‑draft, --spec‑type draft‑dflash, and --spec‑draft‑n‑max 4
• Run two test patterns via the web frontend, clearing session between runs
• Record wall‑clock time, token count, and throughput from llama.cpp logs

证据

DFlash‑2 adds two things on top of the original mechanism: Lightweight Path Selector and Local Convolution

来源

Building llama.cpp from this PR gives you an inference engine that supports DFlash‑2

来源

We used an NVIDIA RTX A4000 GPU for this test

来源