The memory layer that never calls an LLM: what that buys, and what it costs

Описание и применение

Выбор между Mem0 и RE‑call зависит от требований к точности, стоимости и конфиденциальности данных. Mem0 обеспечивает лучшую точность в задачах временного рассуждения, но требует вызовов LLM, токенов и расходов. RE‑call не использует LLM, имеет нулевые расходы и быстрее, но может уступать в точности.

Задача

Choosing between an LLM‑distilling memory layer (Mem0) and a no‑LLM retrieval layer (RE‑call) for building a question‑answering system that balances accuracy, cost, latency, and data privacy.

Результат

Mem0 delivers higher accuracy on temporal reasoning at the cost of LLM calls, tokens, and ingest time, while RE‑call offers zero cost, faster ingest, and local data retention. The decision hinges on accuracy requirements, budget, and privacy constraints.

Рабочий процесс

• 1. Collect conversation turns and decide whether to store raw or distilled memory.
• 2. If using Mem0, send each write to an LLM to distill and store the result in PostgreSQL; if using RE‑call, store raw turns directly.
• 3. Configure optional flags: embedder (bge-small/large, Voyage, OpenAI), reranker (off by default), and entailment abstention (off by default).
• 4. Run queries against the stored memories using the chosen retrieval strategy.
• 5. Measure accuracy on BEAM or similar benchmarks to validate performance.
• 6. Monitor LLM call counts, token usage, and cost to ensure budget compliance.
• 7. Adjust flags or switch architecture based on observed trade‑offs.

Доказательства

Mem0 calls an LLM when you write a memory, and stores the distilled result. RE‑call calls no LLM ever, and stores your raw turns.

Источник

Llm calls to build the memory: 0 for RE‑call, 272 for Mem0; tokens: 0 vs 2.6 M; cost: $0 vs $7.29; ingest wall clock: 67 s vs 288 s.

Источник

Your data never leaves your infrastructure. Local embeddings, the PostgreSQL you already run, and it works offline.

Источник