We Shipped an Optimization Our Own Benchmark Said Not To

Description and use

The team added parallel aggregation to logq, resulting in a 16% slowdown on real data. The -j flag was shipped with disclosure, and an unexplained CI workflow was removed.

Problem

Adding parallelism to a log query tool’s aggregation stage without parallelizing JSON decoding caused a 16% slowdown on real data, yet the feature was shipped.

Outcome

The team shipped a parallel aggregation flag that was slower on real data, but maintained correctness and transparency. They also removed a problematic CI workflow, improving build reliability.

Workflow

• 1. Added -j flag to spawn N goroutines for aggregation.
• 2. Sharded aggregation by hashing group keys to ensure deterministic placement.
• 3. Verified correctness by comparing byte‑identical output between -j 1 and -j 16 on a 2,000‑record fixture.
• 4. Benchmarked on a 76.3 MB file: -j 1 finished in 6.535 s, -j 8 in 7.558 s (16% slower).
• 5. Decided to ship the flag with disclosure of the regression.
• 6. Removed an unexplained CI workflow that failed without clear cause.
• 7. Documented all benchmarks and reproducibility steps in README and make targets.

Evidence

-j 8 (8 parallel workers) measured 16% slower than -j 1 on a real 76.3MB file — shipped anyway, disclosed the moment it was found

Source

We shipped the flag anyway.

Source

CI landed in that same sweep commit, 58da5cf: matrix build across Linux/macOS/Windows, a coverage gate, -race, fuzzing, a reproducibility check.

Source