A question on Discord found two bugs in my RAG system. Fixing them found a third

Description and use

The RAG system mis‑grouped citations, mis‑ranked memory recency, and had a Kafka sink bug. After fixes, citations are grouped by document, memory age uses last spoken time, and the sink reliably handles deletes and retries.

Problem

RAG system misrepresented source citations, mis‑ranked memory recency, and had a Kafka sink bug that silently failed on delete events.

Outcome

Citations now correctly reflect unique documents, memory recency aligns with user interactions, and the Kafka sink reliably ingests changes without silent failures.

Workflow

• Identify duplicate source citations by grouping chunks by document id before prompt construction.
• Change memory age calculation to use last spoken time instead of created_at, updating ranking formula.
• Implement Kafka Connect sink with stable event ids, delete handling, blank row removal, placeholder rejection, and offset safety.
• Test end‑to‑end ingestion of snapshots, updates, deletes, and blanks.
• Verify that recency tests now fail against old query to confirm fix.
• Deploy updated sink and monitor for 5xx failures and DLQ behavior.

Evidence

Retrieval returns chunks, not documents. Ossian took the top six and numbered them for the model…

Source

The query measured it from created_at…

Source

insert or update on table "ingest_events" violates foreign key constraint…

Source