Why your agent's memory layer should make you prove it wrong

说明与用途

使用 nautilus‑compass 和 VerifyPack 进行 AI 代理记忆层可复现基准测试的工作流程。

问题

AI‑agent memory layers often rely on implicit trust and opaque benchmarks, making it hard to verify their performance claims.

结果

The workflow yields a memory layer that achieves higher precision@1 and precision@5 on LongMemEval‑S compared to mem0 2.0.19, while enabling full byte‑level reproducibility of benchmark claims.

工作流

• Clone the nautilus‑compass repo
• Run the install script to set up the plugin
• Start the daemon to enable memory services
• Configure your MCP client to use the local or hosted gateway
• Run benchmark tests to compare against other memory layers
• Use VerifyPack commands to recompute and verify published numbers

证据

P@1 0.890 vs 0.774, P@5 0.978 vs 0.916

来源

Every number we publish, you should be able to recompute from bytes, without trusting us.

来源

Two stdlib-only commands re-derive any number — no third-party dependencies, no "trust our notebook".

来源