When does an agent beat RAG? We benchmarked three pipelines on a TigerGraph knowledge graph

说明与用途

代理式管道在100个公开问题上实现了99%的准确率，远超传统RAG（48%）和GraphRAG（69%）。

问题

Determining when an agentic pipeline outperforms a standard Retrieval-Augmented Generation (RAG) system on a knowledge graph based question answering task.

结果

Agentic GraphRAG achieved 99% accuracy, a 51 percentage point improvement over RAG and a 30 percentage point improvement over GraphRAG, while maintaining reasonable token usage and latency.

工作流

• 1. Build graph from Wikipedia infoboxes and store in TigerGraph. 2. Create three pipelines: RAG (top‑8 chunks), GraphRAG (top‑6 chunks + fixed graph expansion), Agentic GraphRAG (LLM orchestrator selecting tools step‑by‑step). 3. Run 100 public questions through each pipeline. 4. Evaluate answers with an LLM judge and deterministic pre‑check. 5. Record accuracy, tokens, latency, and tokens per correct answer.
• 6. Analyze per‑question‑type performance and identify where agents provide value.
• 7. Document cost and token usage per pipeline.
• 8. Plan next‑step enhancements (ClaimVersion vertex).

证据

The agentic pipeline scored 99% on 100 public questions. Plain RAG scored 48%.

来源

Agentic GraphRAG scored 99% on 100 public questions.

来源

The agent uses 1.24x the tokens of RAG for 2.06x the accuracy. Per correct answer it is cheaper than RAG by 40%.

来源