Your Self-Hosted AI Assistant Keeps Forgetting You. Here Is How We Measured a Fix

Description and use

The article explains how to improve a self-hosted AI assistant’s memory by using SQLite for persistence, hybrid BM25 and vector retrieval with a relevance gate, and nightly maintenance, resulting in higher accuracy and less hallucination.

Problem

A self-hosted AI assistant loses or misremembers user-provided facts after a few weeks of use, leading to repeated questions, incorrect answers, and fabricated responses.

Outcome

Hybrid retrieval with gating improved F1 from 0.38 to 0.48 and reduced confabulation from 0.23 to 0.03 on adversarial questions, while keeping daily model costs low.

Workflow

• 1. Store all memories, relations, and transcripts in one SQLite database in WAL mode.
• 2. Embed dates within memory facts and record rejections to preserve conflicting information.
• 3. When querying, perform BM25 keyword search and dense vector search, merge results, optionally rerank with an LLM.
• 4. Apply a relevance threshold gate; if no results exceed the gate, return no context and let the model say it does not know.
• 5. Run nightly maintenance: dedupe near-duplicates, link related memories, prune obsolete entries.
• 6. Monitor performance using the sqlite3 CLI and benchmark against LoCoMo.
• 7. Adjust thresholds and maintenance schedules based on observed F1 scores.

Evidence

Keep the whole memory store in one file. We use SQLite: memories, relations between memories, and session transcripts all live in one database.

Source

Hybrid retrieval fixes a surprising amount: BM25 keyword search catches the exact name, ticket number, or identifier that embeddings blur away.

Source

The gated system scored 0.97 against 0.77 on adversarial questions with no correct answer stored.

Source

Our estimate from the per-operation cost table is about 5 to 10 cents a day of model spend at 100 messages a day, on top of a $5 to 8 monthly VPS, or nothing extra if you run it on hardware you already have.

Source