Reranking Models: The Cheapest RAG Quality Upgrade

说明与用途

在密集向量检索和生成之间添加交叉编码器重新排序器，将RAG答案准确率从61%提升至84%，成本不到每月40美元。

问题

RAG systems often return irrelevant chunks because bi‑encoder retrieval misses fine‑grained relevance signals, leading to low answer accuracy.

结果

Adding a cheap cross‑encoder reranker between retrieval and generation improved answer accuracy from 61 % to 84 % and cost less than $40 per month.

工作流

• 1. Run bi‑encoder dense vector search to retrieve top 50‑100 candidates. 2. Merge and dedupe the candidate list. 3. Apply a cross‑encoder reranker (e.g., BAAI/bge‑reranker‑v2‑m3) to score each candidate against the query. 4. Keep the top 5‑10 ranked chunks. 5. Pass those chunks to the LLM prompt for generation.

证据

Answer accuracy on our test set went from 61% to 84%

来源

The reranker cost less than $40 that month.

来源

Hybrid retrieval alone moved recall of the correct chunk into the top 50 from 78% to 91%.

来源

Take 10,000 queries per month with 50 candidates reranked per query. With Cohere, each query is one search call since 50 docs fit the 100-doc limit, so 10,000 searches cost $20 per month.

来源