MTP on an RTX 3090: Faster Tokens, but What About Coding Quality?

Description and use

Enabling Multi‑Token Prediction on an RTX 3090 increased token throughput by 56 % and reduced task time by 20–40 %, but did not improve the number of completed tasks or coding quality.

Problem

Assess whether enabling Multi‑Token Prediction (MTP) on an RTX 3090 improves generation speed without degrading coding quality for Qwen3.8‑27B.

Outcome

MTP increased token throughput by 56% and reduced task time by 20–40% on two cache tasks, but did not improve the number of completed tasks or predefined check pass rates. One transfer task produced a patch‑quality difference, but overall coding quality was not degraded.

Workflow

• 1. Configure server with Qwen3.8‑27B Q4_K_M GGUF on RTX 3090.
• 2. Set generation parameters: medium thinking, temp 1.0, top_p 0.95, top_k 20, 8,192‑token limit, 480‑second task budget.
• 3. Run eight paired comparisons (16 attempts) with two seeds (4242, 8675309).
• 4. For each pair, run MTP off (spec-type none) and MTP on (spec-type draft‑mtp).
• 5. Measure aggregate coding‑generation throughput, predefined check pass rates, and task completion times.
• 6. Record detailed task scores and any patch‑quality differences.
• 7. Analyze cache, ledger, and build‑planner scores, and review overflow and boundary cases.
• 8. Summarize speed gains and quality impact.
• 9. Document configuration, request audit, and limits of comparison.

Evidence

Enabling MTP on this RTX 3090 raised generation throughput from 36.0 to 56.2 tokens/s, about 56% faster

Source

The two cache tasks that succeeded in both modes also finished about 20% and 40% sooner

Source

We ran eight paired comparisons, sixteen attempts, using the same model file. Each pair shared its task, seed, and initial environment; only the MTP setting changed.

Source

The entire 35/80 versus 26/80 gap came from one transfer task at one seed.

Source