Three Raft bugs that only existed once I added log compaction

Description and use

Log compaction in a Go Raft implementation revealed three hidden bugs, discovered by a deterministic simulator using virtual time and in‑memory networking.

Problem

Raft implementations can hide subtle bugs behind log redundancy; log compaction exposes them.

Outcome

Three bugs were uncovered: a late AppendEntries message misinterpreted as a conflict, a snapshotting edge case causing state divergence, and a log compaction race leading to inconsistent commit indices. Each bug was reproduced with a single seed and added to the test suite.

Workflow

• 1. Implement Raft core as pure synchronous state machine.
• 2. Replace nondeterministic components with interfaces (time, network, disk, scheduler).
• 3. Build deterministic simulator that runs the core on a single thread with a seeded random source.
• 4. Run extensive fuzz campaigns (10k seeds) and record failures.
• 5. Investigate failures to identify root causes and create regression tests.

Evidence

Adding snapshotting exposed three real bugs.

Source

The simulator runs the real consensus core, but not bbolt and not the node driver.

Source

The event loop is a priority queue ordered by virtual time, and simultaneous events are routine — every node's first tick is at t=0.

Source