I Built a Crash-Safe Key-Value Store Without sqlite3 — Then I Killed It

说明与用途

Anvil 是一个完全使用 Python 标准库构建的 crash‑safe 嵌入式键值存储，展示了如何在没有第三方依赖的情况下管理追加日志、内存索引、校验和和原子压缩。

问题

Build a crash‑safe, embedded key‑value store with no third‑party runtime dependencies, exposing all persistence guarantees in code.

结果

Anvil demonstrates that a fully crash‑safe key‑value store can be built with only the Python standard library, exposing all persistence guarantees and passing rigorous crash‑harness tests.

工作流

• Design append‑only log and binary record format using struct and zlib.crc32.
• Implement in‑memory index mapping keys to log offsets.
• Add single‑writer locking with fcntl/msvcrt.
• Implement crash harness: start writer, perform writes, kill with SIGKILL, reopen store and verify all acknowledged writes are intact.
• Implement compaction: write temporary compacted file, fsync, atomic os.replace, fsync directory.
• Add regression tests for corruption scenarios (e.g., altered length fields).
• Benchmark write/read throughput and database size.
• Publish reproducible single‑file builds via zipfile tooling.

证据

The write‑path crash harness passed 8/8 real SIGKILL iterations.

来源

The benchmark reported 6,899.5 ops/sec write throughput and 200,015.8 ops/sec read throughput.

来源