Synonym Expansion Cannot Lower Recall. That Is Exactly Why Recall Is the Wrong Thing to Tune It On

Описание и применение

Расширение синонимов повышает как полноту, так и точность запросов, опровергая распространенное мнение, что оно только увеличивает полноту. Псевдо‑ответы HyDE показывают низкую эффективность из‑за заполнения общими словами.

Задача

Evaluating query rewriting for Retrieval-Augmented Generation (RAG) pipelines revealed that synonym expansion improves both recall and precision, contradicting the common belief that it only raises recall while lowering precision.

Результат

Synonym expansion improves both recall and precision, challenging the assumption that it only raises recall. HyDE pseudo‑answers perform poorly due to filler terms. Recall is a misleading metric for additive rewrites because it cannot decrease.

Рабочий процесс

• Build BM25 index over real corpus
• Apply query rewrites: verbatim, raw, expand, expandNoisy, hyde, decompose
• Score each rewrite against hand‑labelled relevance
• Compute recall@3, precision@3, MRR, mean query length
• Compare metrics across rewrites
• Analyze effect of synonym quality and query length
• Verify results with dense term‑document matrix
• Document findings

Доказательства

Expansion raised both — recall 0.688 → 0.938 and precision 0.375 → 0.542.

Источник

HyDE lost to doing nothing. MRR 0.938 → 0.323.

Источник

A superset of the query can only retrieve a superset of the documents. Recall cannot fall.

Источник