Tokens arrived on time — the renderer didn't: measuring a 90x display lag in local-LLM streaming (and how Ollama fixed it)

Description and use

Using all CPU cores in a local LLM IDE caused a 90× increase in display lag, even though token generation speed stayed the same. Upgrading to Ollama 0.40.2 fixed the issue because the new engine uses fewer threads and does not saturate the CPU.

Problem

Local-LLM IDEs can feel slower when the model uses all CPU cores, even though token generation speed is unchanged.

Outcome

Using all CPU cores for local LLM inference caused a 90× increase in display lag and a 6× slower UI click response on Ollama 0.21.0. Upgrading to Ollama 0.40.2 reduced the lag dramatically because the new engine defaults to 3 threads and does not saturate the CPU. Thread caps and the think flag remain useful for guaranteeing consistent performance across versions.

Workflow

• Measure token arrival via a proxy between IDE and Ollama.
• Record DOM render time with a MutationObserver.
• Measure event-loop delay with a 100 ms setTimeout drift.
• Count effective FPS with requestAnimationFrame.
• Time a real UI click during generation.
• Run tests on a Ryzen 5 4500U with fixed model settings.
• Compare results for all 6 cores vs 4 cores on Ollama 0.21.0 and 0.40.2.
• Analyze runner process to understand thread usage.
• Deploy thread cap and think flag settings in Teaspoon IDE.
• Publish findings and update IDE settings accordingly.

Evidence

On Ollama 0.21.0, running inference on all 6 cores vs. 4 produced the same token rate but 90x worse display lag (P95 181ms → 2ms) and 6x worse UI click response (929ms → 158ms).

Source

On Ollama 0.40.2 the gap nearly vanished, because the runner internals changed.

Source

The default changed: unspecified now means 3 threads — it never grabs all cores in the first place.

Source