The memory layer that never calls an LLM: what that buys, and what it costs

Description and use

Choosing between Mem0 and RE‑call depends on accuracy, cost, and data privacy needs. Mem0 offers better temporal reasoning accuracy but incurs LLM calls, tokens, and cost. RE‑call has no LLM calls, zero cost, and faster ingest but may be less accurate.

Problem

Choosing between an LLM‑distilling memory layer (Mem0) and a no‑LLM retrieval layer (RE‑call) for building a question‑answering system that balances accuracy, cost, latency, and data privacy.

Outcome

Mem0 delivers higher accuracy on temporal reasoning at the cost of LLM calls, tokens, and ingest time, while RE‑call offers zero cost, faster ingest, and local data retention. The decision hinges on accuracy requirements, budget, and privacy constraints.

Workflow

• 1. Collect conversation turns and decide whether to store raw or distilled memory.
• 2. If using Mem0, send each write to an LLM to distill and store the result in PostgreSQL; if using RE‑call, store raw turns directly.
• 3. Configure optional flags: embedder (bge-small/large, Voyage, OpenAI), reranker (off by default), and entailment abstention (off by default).
• 4. Run queries against the stored memories using the chosen retrieval strategy.
• 5. Measure accuracy on BEAM or similar benchmarks to validate performance.
• 6. Monitor LLM call counts, token usage, and cost to ensure budget compliance.
• 7. Adjust flags or switch architecture based on observed trade‑offs.

Evidence

Mem0 calls an LLM when you write a memory, and stores the distilled result. RE‑call calls no LLM ever, and stores your raw turns.

Source

Llm calls to build the memory: 0 for RE‑call, 272 for Mem0; tokens: 0 vs 2.6 M; cost: $0 vs $7.29; ingest wall clock: 67 s vs 288 s.

Source

Your data never leaves your infrastructure. Local embeddings, the PostgreSQL you already run, and it works offline.

Source