Building a RAG App That Knows When to Say "I Don't Know"

说明与用途

构建了一个 RAG Web 应用，只能从提供的 PDF 文档中回答问题，显示来源并拒绝无关问题。

问题

Large language models answer confidently even when they lack knowledge, leading to ungrounded responses.

结果

The app delivers grounded answers from the study notes, displays source passages, and politely declines unrelated questions, demonstrating effective retrieval quality and prompt design.

工作流

• Chunk the PDF into ~800 character pieces with overlap
• Embed each chunk locally using an open‑source model
• Store embeddings in FAISS for similarity search
• Embed user query and retrieve four nearest chunks
• Send retrieved chunks and strict prompt to Groq LLM to generate answer with JSON output

证据

Large language models are impressive, but they answer confidently even when they have no idea.

来源

RAG works by looking up the relevant text first and then asking the model to answer from it, which keeps answers grounded and checkable.

来源

Ask something unrelated, like the capital of France, and it politely says it could not find that in the documentation.

来源

Chunk size, overlap and a similarity threshold determine whether the app answers correctly or admits it does not know.

来源