Reranking Models: The Cheapest RAG Quality Upgrade

Описание и применение

Добавление кросс‑энкодерного ранжировщика между поиском по векторам и генерацией повышает точность ответов в системах RAG с 61 % до 84 % при минимальных затратах.

Задача

RAG systems often return irrelevant chunks because bi‑encoder retrieval misses fine‑grained relevance signals, leading to low answer accuracy.

Результат

Adding a cheap cross‑encoder reranker between retrieval and generation improved answer accuracy from 61 % to 84 % and cost less than $40 per month.

Рабочий процесс

• 1. Run bi‑encoder dense vector search to retrieve top 50‑100 candidates. 2. Merge and dedupe the candidate list. 3. Apply a cross‑encoder reranker (e.g., BAAI/bge‑reranker‑v2‑m3) to score each candidate against the query. 4. Keep the top 5‑10 ranked chunks. 5. Pass those chunks to the LLM prompt for generation.

Доказательства

Answer accuracy on our test set went from 61% to 84%

Источник

The reranker cost less than $40 that month.

Источник

Hybrid retrieval alone moved recall of the correct chunk into the top 50 from 78% to 91%.

Источник

Take 10,000 queries per month with 50 candidates reranked per query. With Cohere, each query is one search call since 50 docs fit the 100-doc limit, so 10,000 searches cost $20 per month.

Источник