DFlash-2: Benchmarking Z-Lab's Successor to DFlash for Accuracy and Throughput Gains

Описание и применение

DFlash‑2, улучшенный механизм предсказания токенов, повышает пропускную способность и точность по сравнению с оригинальным DFlash на модели Muse Glimmer 30B, используя RTX A4000 и llama.cpp.

Задача

Determine whether DFlash-2 improves throughput and accuracy over DFlash and a baseline model on Muse Glimmer 30B.

Результат

DFlash‑2 achieved higher token throughput than DFlash while reducing the drop‑off in draft acceptance toward the end of a block, demonstrating that the lightweight path selector and local convolution effectively improve draft‑token ordering accuracy.

Рабочий процесс

• Clone llama.cpp and apply PR #27342 for DFlash‑2 support
• Build llama.cpp with CUDA enabled
• Download Muse‑Glimmer‑30B‑UD‑Q2_K_XL.gguf, Muse‑Glimmer‑30B‑DFlash2‑Q4_K_M.gguf, and dflash‑kquant.gguf
• Start llama‑server with --model, --model‑draft, --spec‑type draft‑dflash, and --spec‑draft‑n‑max 4
• Run two test patterns via the web frontend, clearing session between runs
• Record wall‑clock time, token count, and throughput from llama.cpp logs

Доказательства

DFlash‑2 adds two things on top of the original mechanism: Lightweight Path Selector and Local Convolution

Источник

Building llama.cpp from this PR gives you an inference engine that supports DFlash‑2

Источник

We used an NVIDIA RTX A4000 GPU for this test

Источник