The seed is just the oldest record, and other things I got wrong about a YantrikDB merge bug

Description and use

YantrikDB merge bug caused by English-only embedder scoring unrelated Cyrillic texts highly, leading to incorrect clustering. Fixing embedder choice and gist handling resolves the issue.

Problem

Five unrelated memories were incorrectly merged into one consolidated record in YantrikDB, causing misinformation and loss of importance scores.

Outcome

The merge bug was traced to the default English‑only embedder scoring unrelated Cyrillic texts highly, causing improper clustering. Adjusting embedder choice and gist handling resolves the issue.

Workflow

• 1. Issue #253 reported merging unrelated memories.
• 2. Analyzed consolidation logic: importance scaling, cluster size limits, and redundancy triggers.
• 3. Reviewed plugin’s compression gist generation and length caps.
• 4. Tested embeddings with default potion-base and multilingual potion-multilingual-128M models.
• 5. Determined default embedder caused high similarity between unrelated Cyrillic texts.
• 6. Proposed fixes: exclude summaries from clustering, cap gist similarity influence, promote pre_compression tag, cap gist length.
• 7. Planned follow‑ups: expose reembed() to Python, warn on language mismatch, add config switch for embedder.

Evidence

Five memories from four unrelated topics (two delegated-task prompts, a fact about DPI community tools, two Bitwarden facts) were folded into one consolidated record whose text began "Hermes dashboard autostart: TWO mechanisms..."

Source

Over many think() passes (the engine only commits a handful of consolidations per pass by default), the store had built up 84 distinct consolidation targets: {2: 31, 10: 22, 3: 8, 1: 5, 6: 5, 4: 4, 5: 3, 8: 3, 7: 2, 9: 1}

Source

The default, potion-base, is the static, English-only model bundled in the wheel. Unrelated Cyrillic text scores 0.83 to 0.93 against other unrelated Cyrillic text, well over the 0.6 clustering threshold.

Source