I built a local AI memory gate on a CPU, and my 7B model scored worse than my 1.5B model because it was too smart

说明与用途

为本地LLM构建轻量级离线记忆层，采用SQLite、Hebbian可塑性和超维向量；1.5B Qwen取得50%准确率，5.2B Qwen因评测严格而仅25%。

问题

Creating a lightweight, offline memory layer for local LLMs that avoids heavy vector databases and VRAM waste.

结果

The 1.5B Qwen model achieved 50% retrieval accuracy, while the 5.2B Qwen model, despite richer outputs, scored only 25% due to rigid evaluation expectations.

工作流

• 1. Store facts as SQLite triples (Subject-Predicate-Object).
• 2. Track co-occurring concepts via Hebbian plasticity to build associative weights.
• 3. Maintain a 10,000‑dimensional leaky context vector for pronoun resolution and query gating.
• 4. Ingest conversation, extract triples, and update the database.
• 5. Run retrieval accuracy benchmark on Qwen models.

证据

When I ran Qwen 2 (1.5B), it got around 50.0% Retrieval Accuracy. But when I upgraded to the much smarter Qwen 3 (5.2GB), the ingestion time jumped to 20 minutes on my local machine, and its score actually dropped to 25.0%!

来源

The codebase is written in pure Python, is fully open-source (under the AGPL-3.0 copyleft license), and is designed to run entirely offline on consumer hardware.

来源