DFlash-2: Benchmarking Z-Lab's Successor to DFlash for Accuracy and Throughput Gains

Description and use

DFlash‑2 improves throughput and draft‑token ordering accuracy over DFlash on Muse Glimmer 30B, demonstrated with an RTX A4000 and llama.cpp.

Problem

Determine whether DFlash-2 improves throughput and accuracy over DFlash and a baseline model on Muse Glimmer 30B.

Outcome

DFlash‑2 achieved higher token throughput than DFlash while reducing the drop‑off in draft acceptance toward the end of a block, demonstrating that the lightweight path selector and local convolution effectively improve draft‑token ordering accuracy.

Workflow

• Clone llama.cpp and apply PR #27342 for DFlash‑2 support
• Build llama.cpp with CUDA enabled
• Download Muse‑Glimmer‑30B‑UD‑Q2_K_XL.gguf, Muse‑Glimmer‑30B‑DFlash2‑Q4_K_M.gguf, and dflash‑kquant.gguf
• Start llama‑server with --model, --model‑draft, --spec‑type draft‑dflash, and --spec‑draft‑n‑max 4
• Run two test patterns via the web frontend, clearing session between runs
• Record wall‑clock time, token count, and throughput from llama.cpp logs

Evidence

DFlash‑2 adds two things on top of the original mechanism: Lightweight Path Selector and Local Convolution

Source

Building llama.cpp from this PR gives you an inference engine that supports DFlash‑2

Source

We used an NVIDIA RTX A4000 GPU for this test

Source