EAGLE-4 does not run on a Mac. I measured MTP instead: +92% on one model, -19% on another

说明与用途

MTP 在 Qwen3.8‑27B 上将代码生成速度提升 92%，但在 Ornith‑1.5‑35B‑A3B 上将散文生成速度降低 19%。

问题

Determine if speculative decoding method EAGLE-4 improves inference speed on Apple Silicon, and compare with existing MTP technique.

结果

MTP significantly accelerates dense Qwen3.8‑27B code generation (≈1.92×) but slightly degrades MoE Ornith‑1.5‑35B‑A3B prose generation (≈0.81×). Prefill times are unaffected or slightly slower with MTP. Concurrency benefits are mixed: dense model gains up to 1.73× with four requests, while MoE model suffers slowdown.

工作流

• 1. Set up Mac Studio with Apple M2 Ultra, 64 GB unified memory, macOS 26.5.
• 2. Install oMLX 0.7.1rc1 and load models: Qwen3.8‑27B oQ4e‑mtp, Ornith‑1.5‑35B‑A3B oQ4e‑mtp, and Qwen3‑4B‑Instruct‑2507 4‑bit.
• 3. Run each model in four phases: MTP off, MTP on, MTP off, MTP on, with fresh server per phase.
• 4. Use temperature 0, streaming, one request at a time, no prefix cache, no memory guard.
• 5. Execute 10 runs per prompt per phase (Kotlin coding task and 350‑word prose), 512 max tokens, warm‑up runs excluded.
• 6. Record decode tokens per second, prefill time, and concurrency metrics.
• 7. Analyze results: compute mean tok/s, on/off ratios, parked runs, and compare across models and scenarios.

证据

EAGLE-4 as released does not run on a Mac.

来源

On a dense 27B model, MTP made code generation 92% faster (thinking off, one request).

来源

On a 35B mixture-of-experts (MoE) model, MTP made prose generation 19% slower (thinking off, one request).

来源

The control model, Qwen3-4B, ran at 159.2 to 159.9 tok/s in all four scenarios.

来源