MTP on an RTX 3090: Faster Tokens, but What About Coding Quality?

说明与用途

在 RTX 3090 上启用 Multi‑Token Prediction 提升了 56% 的生成吞吐量，并将任务时间缩短 20–40%，但未提升完成任务数量或代码质量。

问题

Assess whether enabling Multi‑Token Prediction (MTP) on an RTX 3090 improves generation speed without degrading coding quality for Qwen3.8‑27B.

结果

MTP increased token throughput by 56% and reduced task time by 20–40% on two cache tasks, but did not improve the number of completed tasks or predefined check pass rates. One transfer task produced a patch‑quality difference, but overall coding quality was not degraded.

工作流

• 1. Configure server with Qwen3.8‑27B Q4_K_M GGUF on RTX 3090.
• 2. Set generation parameters: medium thinking, temp 1.0, top_p 0.95, top_k 20, 8,192‑token limit, 480‑second task budget.
• 3. Run eight paired comparisons (16 attempts) with two seeds (4242, 8675309).
• 4. For each pair, run MTP off (spec-type none) and MTP on (spec-type draft‑mtp).
• 5. Measure aggregate coding‑generation throughput, predefined check pass rates, and task completion times.
• 6. Record detailed task scores and any patch‑quality differences.
• 7. Analyze cache, ledger, and build‑planner scores, and review overflow and boundary cases.
• 8. Summarize speed gains and quality impact.
• 9. Document configuration, request audit, and limits of comparison.

证据

Enabling MTP on this RTX 3090 raised generation throughput from 36.0 to 56.2 tokens/s, about 56% faster

来源

The two cache tasks that succeeded in both modes also finished about 20% and 40% sooner

来源

We ran eight paired comparisons, sixteen attempts, using the same model file. Each pair shared its task, seed, and initial environment; only the MTP setting changed.

来源

The entire 35/80 versus 26/80 gap came from one transfer task at one seed.

来源