Why a Lock‑Free Ring Buffer Beats Mutex Queues in a 200 kRPS Rust Service

Description and use

In a telemetry ingest service at 200 kRPS, a Mutex became a bottleneck. Switching to a lock‑free ring buffer reduced 95th‑percentile latency from 1.8 ms to 0.6 ms and eliminated spikes, improving predictability.

Problem

A telemetry ingest service hit 200 k requests per second on a single 8‑core box; a Mutex<VecDeque<T>> shared between an async dispatcher and worker threads became a hotspot, inflating tail latency and causing occasional lock‑starvation spikes.

Outcome

The lock‑free ring buffer cut 95th‑percentile latency from 1.8 ms to 0.6 ms and removed lock‑starvation spikes, improving predictability for a 200 kRPS telemetry ingest service.

Workflow

• 1. Profile the service with perf to identify contention hotspots.
• 2. Replace Mutex<VecDeque<T>> with a custom RingBuffer<T> using atomic head/tail indices.
• 3. Ensure the buffer size is a power of two and bounded to handle burst traffic.
• 4. Use Relaxed ordering for payloads and Acquire/Release for indices to avoid cache‑line ping‑pong.
• 5. Benchmark the new implementation on the same hardware.
• 6. Validate that latency improves and spikes disappear.

Evidence

In a telemetry ingest service we were hitting 200 k requests per second on a single 8‑core box. The naive design used a Mutex<VecDeque<T>> shared between the async dispatcher and a pool of worker threads.

Source

Profiling with perf showed the mutex lock function consuming ~12 % of total CPU cycles, dwarfing the actual message‑processing work.

Source

Benchmarks on the same hardware dropped the per‑request latency from 1.8 ms (95th percentile) to 0.6 ms and eliminated the periodic latency spikes.

Source