Making Qwen Faster on an RTX 3090

Description and use

Speeding up Qwen3.8‑27B generation on an RTX 3090 using IQ3_S weights, Q8 KV cache, and MTP4 raised throughput to ~60 tokens/s while keeping similar pass rates.

Problem

Improve generation speed of Qwen3.8-27B on a single 24 GB RTX 3090 while maintaining functional and delivered pass rates for coding repair tasks.

Outcome

Using IQ3_S weights, Q8 KV cache, and MTP4 on a single RTX 3090 increased generation speed from ~33 tokens/s to ~60 tokens/s (≈80 % faster) while maintaining comparable delivered pass rates (21/24). Ledger task failures were traced to output truncation and test interface defects, which were corrected to achieve full pass rates.

Workflow

• 1. Select Qwen3.8-27B model and quantization bundle (IQ3_S with MTP head).
• 2. Configure KV cache to Q8 format and set total context window to 128 K tokens.
• 3. Enable MTP4 (max 4 draft tokens) and set output cap to 64 K tokens.
• 4. Run eight repair tasks (six Python, two TypeScript) each once, measuring generation rate and delivered pass counts.
• 5. Compare against baseline (Q4_K_M, MTP off) and alternative (Bartowski IQ4_XS).
• 6. Adjust output cap for Ledger task to 64 K to resolve truncation.
• 7. Validate functional and delivered passes, noting any test defects.
• 8. Record GPU memory usage and task times.
• 9. Optionally test larger 220 K context window for long-input migration task.
• 10. Analyze trade-offs between speed, memory, and correctness.

Evidence

Generation speed during coding tasks increased from roughly 33 tokens/s with the original configuration to around 60 tokens/s.

Source

IQ3_S quantized weights, Q8 KV cache, and MTP4

Source

The IQ3 bundle generated approximately 80% faster than the original Q4 bundle.

Source

MTP4 was approximately 5.7% faster than MTP2; greater depth added no benefit.

Source