Synonym Expansion Cannot Lower Recall. That Is Exactly Why Recall Is the Wrong Thing to Tune It On

Description and use

Synonym expansion improves both recall and precision, refuting the belief that it only raises recall. HyDE pseudo‑answers perform poorly due to filler terms.

Problem

Evaluating query rewriting for Retrieval-Augmented Generation (RAG) pipelines revealed that synonym expansion improves both recall and precision, contradicting the common belief that it only raises recall while lowering precision.

Outcome

Synonym expansion improves both recall and precision, challenging the assumption that it only raises recall. HyDE pseudo‑answers perform poorly due to filler terms. Recall is a misleading metric for additive rewrites because it cannot decrease.

Workflow

• Build BM25 index over real corpus
• Apply query rewrites: verbatim, raw, expand, expandNoisy, hyde, decompose
• Score each rewrite against hand‑labelled relevance
• Compute recall@3, precision@3, MRR, mean query length
• Compare metrics across rewrites
• Analyze effect of synonym quality and query length
• Verify results with dense term‑document matrix
• Document findings

Evidence

Expansion raised both — recall 0.688 → 0.938 and precision 0.375 → 0.542.

Source

HyDE lost to doing nothing. MRR 0.938 → 0.323.

Source

A superset of the query can only retrieve a superset of the documents. Recall cannot fall.

Source