Where retrieval assistants leak: the title, the summary and the cache

Description and use

Retrieval-augmented assistants can leak information through citation titles, summaries written from inaccessible passages, and cached summaries tied to a first user's permissions.

Problem

Retrieval-augmented assistants can disclose information through citation titles, summaries written from retrieved passages, and cached summaries tied to a first user's permissions, even when the assistant is only allowed to answer from documents it fetched.

Outcome

The assistant leaks information through three channels: citation titles (which reveal file names and case types), summaries written from passages the user could not see, and cached summaries tied to a first user's permissions. A safe response should behave as if the record were not there.

Workflow

• Sign in as the lowest-privilege user the tool serves
• Ask a question whose best answer lives in a record that user should never see
• Read the answer
• Read every citation, including titles
• Read the refusal, if there is one

Evidence

Picture a compliance assistant at a bank. A branch employee asks how wire escalations are handled. The answer is accurate and harmless. The first citation reads 'Case file: Hartwell Holdings, sanctions review.'

Source

If the search pulled in passages the user could not see, the model will use them. Filtering the final answer afterwards rarely works, because the passage has already been blended into the prose.

Source

Some systems store generated summaries so the next question is cheaper. Others build the search index using one service account's view of the documents. In both cases, a later user can get an answer shaped by the first user's permissions.

Source

That last one matters more than it looks. A message like 'I can't show you the Hartwell file' tells the user the file exists. A safe response behaves as if the record were not there.

Source