When does an agent beat RAG? We benchmarked three pipelines on a TigerGraph knowledge graph

Description and use

The agentic pipeline achieves 99 % accuracy on 100 public questions, far surpassing standard RAG (48 %) and GraphRAG (69 %).

Problem

Determining when an agentic pipeline outperforms a standard Retrieval-Augmented Generation (RAG) system on a knowledge graph based question answering task.

Outcome

Agentic GraphRAG achieved 99% accuracy, a 51 percentage point improvement over RAG and a 30 percentage point improvement over GraphRAG, while maintaining reasonable token usage and latency.

Workflow

• 1. Build graph from Wikipedia infoboxes and store in TigerGraph. 2. Create three pipelines: RAG (top‑8 chunks), GraphRAG (top‑6 chunks + fixed graph expansion), Agentic GraphRAG (LLM orchestrator selecting tools step‑by‑step). 3. Run 100 public questions through each pipeline. 4. Evaluate answers with an LLM judge and deterministic pre‑check. 5. Record accuracy, tokens, latency, and tokens per correct answer.
• 6. Analyze per‑question‑type performance and identify where agents provide value.
• 7. Document cost and token usage per pipeline.
• 8. Plan next‑step enhancements (ClaimVersion vertex).

Evidence

The agentic pipeline scored 99% on 100 public questions. Plain RAG scored 48%.

Source

Agentic GraphRAG scored 99% on 100 public questions.

Source

The agent uses 1.24x the tokens of RAG for 2.06x the accuracy. Per correct answer it is cheaper than RAG by 40%.

Source