vLLM vs SGLang: PagedAttention vs RadixAttention Benchmarks

Description and use

<blockquote> <p><strong>Direct Answer:</strong> In our empirical benchmarks on multi-turn agent workflows, SGLang's RadixAttention outperformed vLLM's PagedAttention Automatic Prefix Caching (APC), slashing warm Time-To-First-Token (TTFT) from 184ms to 68ms (an 82.9% latency reduction over cold prefill) and boosting sustained throughput by 39.8%. While vLLM remains superior for broad hardware support and speculative decoding, SGLang is currently the de facto serving engine for agentic tool loops.</p> </blockquote> <h2> Why Agentic AI Workloads Break Traditional LLM Serving </h2> <p><strong>Tra

Problem

<blockquote> <p><strong>Direct Answer:</strong> In our empirical benchmarks on multi-turn agent workflows, SGLang's RadixAttention outperformed vLLM's PagedAttention Automatic Prefix Caching (APC), slashing warm Time-To-First-Token (TTFT) from 184ms to 68ms (an 82.9% latency reduction over cold prefill) and boosting sustained throughput by 39.8%. While vLLM remains superior for broad hardware support and speculative decoding, SGLang is currently the de facto serving engine for agentic tool loops.</p> </blockquote> <h2> Why Agentic AI Workloads Break Traditional LLM Serving </h2> <p><strong>Tra

Outcome

<blockquote> <p><strong>Direct Answer:</strong> In our empirical benchmarks on multi-turn agent workflows, SGLang's RadixAttention outperformed vLLM's PagedAttention Automatic Prefix Caching (APC), slashing warm Time-To-First-Token (TTFT) from 184ms to 68ms (an 82.9% latency reduction over cold prefill) and boosting sustained throughput by 39.8%. While vLLM remains superior for broad hardware support and speculative decoding, SGLang is currently the de facto serving engine for agentic tool loops.</p> </blockquote> <h2> Why Agentic AI Workloads Break Traditional LLM Serving </h2> <p><strong>Tra

Evidence

vLLM vs SGLang: PagedAttention vs RadixAttention Benchmarks

Source