My RAG tool refused to answer a question my notes clearly answered

说明与用途

自定义 RAG 系统因文本块被中途截断和短段被稀释而无法检索答案。通过软切分、最佳块大小（约350字符）和拒绝提示，系统能够正确回答或适当拒绝。

问题

A retrieval‑augmented generation (RAG) system built from personal notes failed to retrieve the correct answer because of chunking issues and silent failures.

结果

After adjusting chunk boundaries and adding a refusal prompt, the RAG system correctly answers questions found in the notes and appropriately refuses when no answer exists.

工作流

• 1. Tokenize notes into 500‑character chunks with 100‑character overlap.
• 2. Observe retrieval returns mid‑word fragments, causing no match.
• 3. Increase chunk size to 900 characters, leading to diluted embeddings for short sections.
• 4. Experiment with chunk size 350 and overlap 70, matching whole sections.
• 5. Add a soft cut function to avoid mid‑word splits.
• 6. Test with real and unanswerable questions, noting similarity scores.
• 7. Replace hard relevance floor with a prompt that allows the model to refuse when no answer is found.
• 8. Deploy updated pipeline and verify correct behavior.

证据

I built a retrieval-augmented generation tool over my own notes — no LangChain, no LlamaIndex, every stage by hand.

来源

I can't find the answer to that in your notes.

来源

The answer was there. I could point at the line.

来源

My 500-character cut had landed mid-word. Half the sentence in chunk N, half in chunk N+1, and neither half complete enough to match anything.

来源

The fix is to stop treating chunk size as a hard boundary — scan backwards for somewhere sensible to cut.

来源

At 0.35 the bad question is silenced and two real ones drop to two passages. At 0.40 they return nothing.

来源

What works instead is the prompt — giving the model an acceptable way out: 3. If the passages do not contain the answer, say so plainly...

来源