I built a local AI memory gate on a CPU, and my 7B model scored worse than my 1.5B model because it was too smart

Описание и применение

Создана легковесная офлайн‑память для локальных LLM, использующая SQLite, геббинианскую пластичность и гиперпространственные векторы; 1.5‑млрдовая модель Qwen достигла 50% точности, а 5.2‑млрдовая — 25% из‑за жесткого теста.

Задача

Creating a lightweight, offline memory layer for local LLMs that avoids heavy vector databases and VRAM waste.

Результат

The 1.5B Qwen model achieved 50% retrieval accuracy, while the 5.2B Qwen model, despite richer outputs, scored only 25% due to rigid evaluation expectations.

Рабочий процесс

• 1. Store facts as SQLite triples (Subject-Predicate-Object).
• 2. Track co-occurring concepts via Hebbian plasticity to build associative weights.
• 3. Maintain a 10,000‑dimensional leaky context vector for pronoun resolution and query gating.
• 4. Ingest conversation, extract triples, and update the database.
• 5. Run retrieval accuracy benchmark on Qwen models.

Доказательства

When I ran Qwen 2 (1.5B), it got around 50.0% Retrieval Accuracy. But when I upgraded to the much smarter Qwen 3 (5.2GB), the ingestion time jumped to 20 minutes on my local machine, and its score actually dropped to 25.0%!

Источник

The codebase is written in pure Python, is fully open-source (under the AGPL-3.0 copyleft license), and is designed to run entirely offline on consumer hardware.

Источник