Lost in the Middle: Find Your LLM's Context Dead Zone and Fix It Before the Call

Description and use

<p>One result from the "Lost in the Middle" paper should change how you build RAG prompts. GPT-3.5-Turbo was given 20 retrieved documents with the answer placed in the middle of them, and it scored lower than when it got no documents at all. Answering from memory beat answering with the documents. Here the extra context made the answer worse.</p> <p>Liu et al. (Stanford, UC Berkeley, Samaya AI, 2023) tested multi-document QA and key-value retrieval. They moved the relevant passage from the first slot to the last and tracked accuracy. The result was a U-shaped curve: strong at both edges, down

Problem

<p>One result from the "Lost in the Middle" paper should change how you build RAG prompts. GPT-3.5-Turbo was given 20 retrieved documents with the answer placed in the middle of them, and it scored lower than when it got no documents at all. Answering from memory beat answering with the documents. Here the extra context made the answer worse.</p> <p>Liu et al. (Stanford, UC Berkeley, Samaya AI, 2023) tested multi-document QA and key-value retrieval. They moved the relevant passage from the first slot to the last and tracked accuracy. The result was a U-shaped curve: strong at both edges, down

Outcome

<p>One result from the "Lost in the Middle" paper should change how you build RAG prompts. GPT-3.5-Turbo was given 20 retrieved documents with the answer placed in the middle of them, and it scored lower than when it got no documents at all. Answering from memory beat answering with the documents. Here the extra context made the answer worse.</p> <p>Liu et al. (Stanford, UC Berkeley, Samaya AI, 2023) tested multi-document QA and key-value retrieval. They moved the relevant passage from the first slot to the last and tracked accuracy. The result was a U-shaped curve: strong at both edges, down

Evidence

Lost in the Middle: Find Your LLM's Context Dead Zone and Fix It Before the Call

Source