A question on Discord found two bugs in my RAG system. Fixing them found a third

Описание и применение

Система RAG неправильно группировала источники, некорректно рассчитывала актуальность памяти и имела ошибку в Kafka‑sink. После исправлений источники группируются по документу, возраст памяти учитывает последнее упоминание, а sink корректно обрабатывает удалённые записи.

Задача

RAG system misrepresented source citations, mis‑ranked memory recency, and had a Kafka sink bug that silently failed on delete events.

Результат

Citations now correctly reflect unique documents, memory recency aligns with user interactions, and the Kafka sink reliably ingests changes without silent failures.

Рабочий процесс

• Identify duplicate source citations by grouping chunks by document id before prompt construction.
• Change memory age calculation to use last spoken time instead of created_at, updating ranking formula.
• Implement Kafka Connect sink with stable event ids, delete handling, blank row removal, placeholder rejection, and offset safety.
• Test end‑to‑end ingestion of snapshots, updates, deletes, and blanks.
• Verify that recency tests now fail against old query to confirm fix.
• Deploy updated sink and monitor for 5xx failures and DLQ behavior.

Доказательства

Retrieval returns chunks, not documents. Ossian took the top six and numbered them for the model…

Источник

The query measured it from created_at…

Источник

insert or update on table "ingest_events" violates foreign key constraint…

Источник