We Shipped an Optimization Our Own Benchmark Said Not To

Описание и применение

Команда добавила параллельную агрегацию в инструмент logq, но это привело к 16 % замедлению на реальных данных. Флаг -j был опубликован с раскрытием проблемы, а также удалён некорректный CI‑workflow.

Задача

Adding parallelism to a log query tool’s aggregation stage without parallelizing JSON decoding caused a 16% slowdown on real data, yet the feature was shipped.

Результат

The team shipped a parallel aggregation flag that was slower on real data, but maintained correctness and transparency. They also removed a problematic CI workflow, improving build reliability.

Рабочий процесс

• 1. Added -j flag to spawn N goroutines for aggregation.
• 2. Sharded aggregation by hashing group keys to ensure deterministic placement.
• 3. Verified correctness by comparing byte‑identical output between -j 1 and -j 16 on a 2,000‑record fixture.
• 4. Benchmarked on a 76.3 MB file: -j 1 finished in 6.535 s, -j 8 in 7.558 s (16% slower).
• 5. Decided to ship the flag with disclosure of the regression.
• 6. Removed an unexplained CI workflow that failed without clear cause.
• 7. Documented all benchmarks and reproducibility steps in README and make targets.

Доказательства

-j 8 (8 parallel workers) measured 16% slower than -j 1 on a real 76.3MB file — shipped anyway, disclosed the moment it was found

Источник

We shipped the flag anyway.

Источник

CI landed in that same sweep commit, 58da5cf: matrix build across Linux/macOS/Windows, a coverage gate, -race, fuzzing, a reproducibility check.

Источник