Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

From 47 to 84: The Full LoCoMo Benchmark Record of the NylonME Memory Engine

Description and use

By iteratively adding semantic embeddings, structured writes, adaptive diffusion, full‑run validation, and vector reranking, the team increased LoCoMo evidence‑recall recall@10 from 47.1% to 84.6% and seed‑layer recall to >90%, placing NylonME in the first tier of open‑source memory engines.

Problem

Achieving high evidence-recall recall@10 in an open‑source memory engine while maintaining reproducibility and transparency.

Outcome

By iteratively adding semantic embeddings, structured writes, adaptive diffusion, full‑run validation, and vector reranking, the team increased LoCoMo evidence‑recall recall@10 from 47.1% to 84.6% and seed‑layer recall to >90%, placing NylonME in the first tier of open‑source memory engines.

Workflow

  • 1. Add bge‑m3 embeddings to create a dual‑channel retrieval system (lexical + vector).
  • 2. Implement dual‑layer writes: store raw text in leaf layer and distilled facts in abstract layer.
  • 3. Add adaptive diffusion depth: use max_hops=0 for single‑hop queries, max_hops=4 for multi‑hop queries.
  • 4. Run full‑run validation on 10 sessions (1,536 QA pairs) to eliminate small‑sample bias.
  • 5. Apply vector reranking: compute cosine similarity between query vector and activated nodes, blend with resonance score to improve ranking.

Evidence

LoCoMo evidence‑recall recall@10 = 47.1%

Source

After adding bge‑m3 embeddings, recall jumped from 47.1% to 70.6%

Source

Dual‑layer writes improved recall from 70.6% to 79.2%

Source

Adaptive diffusion depth increased recall to 80.1%

Source

Full‑run validation raised recall to 84.2%

Source

Vector reranking pushed final recall to 84.6%

Source

  • Record date: 2026-09-15T02:31:02.046Z