Building a Policy Q&A System: Architecture, RAG, and Practical Lessons

说明与用途

基于RAG的聊天机器人提供简明的政策答案，并附上来源引用，提升信任度并减少支持呼叫。

问题

Policy documents are publicly available but unreadable for citizens, leading to support calls and guesswork.

结果

The system delivers plain‑language policy answers with source citations and a confidence threshold, improving trust and reducing support calls.

工作流

• User asks a question via React UI
• Express API forwards query to LangChain retrieval
• LangChain searches PostgreSQL + pgvector for relevant policy chunks
• LLM generates answer grounded in retrieved chunks
• System attaches source citations and confidence score
• If confidence below threshold, system replies "I couldn't find a reliable answer"

证据

Making citations a required part of every answer, not an optional extra, forced better retrieval quality from the start

来源

The confidence threshold, even a simple version of it, cut down on confident‑sounding wrong answers a lot

来源