Why a Lock‑Free Ring Buffer Beats Mutex Queues in a 200 kRPS Rust Service

Описание и применение

В сервисе сбора телеметрии при 200 к запросов в секунду замедление вызвано мьютексом, который становился узким местом. Замена на блокирующую кольцевую очередь с атомарными индексами сократила 95‑й процентиль задержки с 1,8 мс до 0,6 мс и устранила пики, улучшив предсказуемость.

Задача

A telemetry ingest service hit 200 k requests per second on a single 8‑core box; a Mutex<VecDeque<T>> shared between an async dispatcher and worker threads became a hotspot, inflating tail latency and causing occasional lock‑starvation spikes.

Результат

The lock‑free ring buffer cut 95th‑percentile latency from 1.8 ms to 0.6 ms and removed lock‑starvation spikes, improving predictability for a 200 kRPS telemetry ingest service.

Рабочий процесс

• 1. Profile the service with perf to identify contention hotspots.
• 2. Replace Mutex<VecDeque<T>> with a custom RingBuffer<T> using atomic head/tail indices.
• 3. Ensure the buffer size is a power of two and bounded to handle burst traffic.
• 4. Use Relaxed ordering for payloads and Acquire/Release for indices to avoid cache‑line ping‑pong.
• 5. Benchmark the new implementation on the same hardware.
• 6. Validate that latency improves and spikes disappear.

Доказательства

In a telemetry ingest service we were hitting 200 k requests per second on a single 8‑core box. The naive design used a Mutex<VecDeque<T>> shared between the async dispatcher and a pool of worker threads.

Источник

Profiling with perf showed the mutex lock function consuming ~12 % of total CPU cycles, dwarfing the actual message‑processing work.

Источник

Benchmarks on the same hardware dropped the per‑request latency from 1.8 ms (95th percentile) to 0.6 ms and eliminated the periodic latency spikes.

Источник