When does an agent beat RAG? We benchmarked three pipelines on a TigerGraph knowledge graph

Описание и применение

Показано, что агентический пайплайн достигает 99 % точности в задаче вопросов по олимпийским событиям, значительно превосходя традиционный RAG (48 %) и GraphRAG (69 %).

Задача

Determining when an agentic pipeline outperforms a standard Retrieval-Augmented Generation (RAG) system on a knowledge graph based question answering task.

Результат

Agentic GraphRAG achieved 99% accuracy, a 51 percentage point improvement over RAG and a 30 percentage point improvement over GraphRAG, while maintaining reasonable token usage and latency.

Рабочий процесс

• 1. Build graph from Wikipedia infoboxes and store in TigerGraph. 2. Create three pipelines: RAG (top‑8 chunks), GraphRAG (top‑6 chunks + fixed graph expansion), Agentic GraphRAG (LLM orchestrator selecting tools step‑by‑step). 3. Run 100 public questions through each pipeline. 4. Evaluate answers with an LLM judge and deterministic pre‑check. 5. Record accuracy, tokens, latency, and tokens per correct answer.
• 6. Analyze per‑question‑type performance and identify where agents provide value.
• 7. Document cost and token usage per pipeline.
• 8. Plan next‑step enhancements (ClaimVersion vertex).

Доказательства

The agentic pipeline scored 99% on 100 public questions. Plain RAG scored 48%.

Источник

Agentic GraphRAG scored 99% on 100 public questions.

Источник

The agent uses 1.24x the tokens of RAG for 2.06x the accuracy. Per correct answer it is cheaper than RAG by 40%.

Источник