How I Built a RAG Search Engine for 10,000 Documents — and Why I Eventually Dropped Answer Generation

说明与用途

为10,000份宗教文档构建的RAG搜索引擎最初返回语义相似但不相关的结果。通过混合稠密-稀疏检索、余弦相似度过滤和Cohere Rerank提升了检索准确性。

问题

Retrieval in a specialized Russian‑language religious corpus produced semantically similar but context‑irrelevant results, undermining user trust.

结果

The search engine now reliably returns documents that truly answer user queries, avoiding misleading summaries and maintaining user trust.

工作流

• Convert JSON to JSONL with per‑record cleanup
• Chunk each record into title‑question and answer‑footnote
• Generate OpenAI text‑embedding‑3‑large vectors
• Store chunks in Qdrant with deterministic IDs
• For a query: normalize text, embed, retrieve dense and sparse results
• Fuse rankings with Reciprocal Rank Fusion
• Filter candidates by cosine similarity thresholds (0.32/0.20)
• Rerank candidates with Cohere Rerank
• Apply final relevance threshold (0.09)
• Return ranked documents or empty set if none meet threshold

证据

Semantic similarity doesn’t necessarily mean a document answers the question.

来源

I introduced an additional cosine similarity check using the dense vectors.

来源

I also introduced another relevance threshold after reranking.

来源