From 47 to 84: The Full LoCoMo Benchmark Record of the NylonME Memory Engine
Description and use
By iteratively adding semantic embeddings, structured writes, adaptive diffusion, full‑run validation, and vector reranking, the team increased LoCoMo evidence‑recall recall@10 from 47.1% to 84.6% and seed‑layer recall to >90%, placing NylonME in the first tier of open‑source memory engines.
Problem
Achieving high evidence-recall recall@10 in an open‑source memory engine while maintaining reproducibility and transparency.
Outcome
By iteratively adding semantic embeddings, structured writes, adaptive diffusion, full‑run validation, and vector reranking, the team increased LoCoMo evidence‑recall recall@10 from 47.1% to 84.6% and seed‑layer recall to >90%, placing NylonME in the first tier of open‑source memory engines.
Workflow
- 1. Add bge‑m3 embeddings to create a dual‑channel retrieval system (lexical + vector).
- 2. Implement dual‑layer writes: store raw text in leaf layer and distilled facts in abstract layer.
- 3. Add adaptive diffusion depth: use max_hops=0 for single‑hop queries, max_hops=4 for multi‑hop queries.
- 4. Run full‑run validation on 10 sessions (1,536 QA pairs) to eliminate small‑sample bias.
- 5. Apply vector reranking: compute cosine similarity between query vector and activated nodes, blend with resonance score to improve ranking.
Evidence
LoCoMo evidence‑recall recall@10 = 47.1%
After adding bge‑m3 embeddings, recall jumped from 47.1% to 70.6%
Dual‑layer writes improved recall from 70.6% to 79.2%
Adaptive diffusion depth increased recall to 80.1%
Full‑run validation raised recall to 84.2%
Vector reranking pushed final recall to 84.6%
- Record date: 2026-09-15T02:31:02.046Z