Building a RAG App That Knows When to Say "I Don't Know"

Description and use

A RAG web app that answers only from a provided PDF, displays source passages, and refuses unrelated queries.

Problem

Large language models answer confidently even when they lack knowledge, leading to ungrounded responses.

Outcome

The app delivers grounded answers from the study notes, displays source passages, and politely declines unrelated questions, demonstrating effective retrieval quality and prompt design.

Workflow

• Chunk the PDF into ~800 character pieces with overlap
• Embed each chunk locally using an open‑source model
• Store embeddings in FAISS for similarity search
• Embed user query and retrieve four nearest chunks
• Send retrieved chunks and strict prompt to Groq LLM to generate answer with JSON output

Evidence

Large language models are impressive, but they answer confidently even when they have no idea.

Source

RAG works by looking up the relevant text first and then asking the model to answer from it, which keeps answers grounded and checkable.

Source

Ask something unrelated, like the capital of France, and it politely says it could not find that in the documentation.

Source

Chunk size, overlap and a similarity threshold determine whether the app answers correctly or admits it does not know.

Source