My RAG tool refused to answer a question my notes clearly answered

Описание и применение

Система RAG, построенная из личных заметок, не находила ответ, потому что куски текста обрезались посередине, а короткие разделы смешивались с большим контекстом. После внедрения мягкого разреза, оптимального размера куска (~350 символов) и инструкции модели отказаться, система корректно отвечала или сообщала об отсутствии ответа.

Задача

A retrieval‑augmented generation (RAG) system built from personal notes failed to retrieve the correct answer because of chunking issues and silent failures.

Результат

After adjusting chunk boundaries and adding a refusal prompt, the RAG system correctly answers questions found in the notes and appropriately refuses when no answer exists.

Рабочий процесс

• 1. Tokenize notes into 500‑character chunks with 100‑character overlap.
• 2. Observe retrieval returns mid‑word fragments, causing no match.
• 3. Increase chunk size to 900 characters, leading to diluted embeddings for short sections.
• 4. Experiment with chunk size 350 and overlap 70, matching whole sections.
• 5. Add a soft cut function to avoid mid‑word splits.
• 6. Test with real and unanswerable questions, noting similarity scores.
• 7. Replace hard relevance floor with a prompt that allows the model to refuse when no answer is found.
• 8. Deploy updated pipeline and verify correct behavior.

Доказательства

I built a retrieval-augmented generation tool over my own notes — no LangChain, no LlamaIndex, every stage by hand.

Источник

I can't find the answer to that in your notes.

Источник

The answer was there. I could point at the line.

Источник

My 500-character cut had landed mid-word. Half the sentence in chunk N, half in chunk N+1, and neither half complete enough to match anything.

Источник

The fix is to stop treating chunk size as a hard boundary — scan backwards for somewhere sensible to cut.

Источник

At 0.35 the bad question is silenced and two real ones drop to two passages. At 0.40 they return nothing.

Источник

What works instead is the prompt — giving the model an acceptable way out: 3. If the passages do not contain the answer, say so plainly...

Источник