When to Use Graphs in RAG: Findings from the ICLR 2026 GraphRAG-Bench Study
GraphResearchgraphragragknowledge-graphsllmretrievalbenchmarkiclrresearch
Loading…
Despite GraphRAG's conceptual promise, recent studies have shown that it frequently underperforms vanilla vector RAG on many real-world tasks. The ICLR 2026 paper "When to Use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation" by Xiang et al. tackles this contradiction head-on, introducing GraphRAG-Bench to systematically determine where graph structures provide measurable benefits.
Is GraphRAG really effective, and in which scenarios do graph structures provide measurable improvements over standard vector retrieval? The paper's contribution is a benchmark designed to answer this precisely: GraphRAG-Bench evaluates models across four task categories of increasing difficulty, spanning fact retrieval, complex reasoning, contextual summarisation, and creative generation.
| Task Category | Example | Graph RAG Benefit |
|---|---|---|
| Fact retrieval | "Who founded company X?" | Minimal — single-hop facts are well-served by vector search |
| Multi-hop reasoning | "Which products are affected by regulation Y via supplier Z?" | Significant — graph traversal captures path dependencies |
| Contextual summarisation | "Summarise the compliance exposure across all EU subsidiaries" | High — community summaries aggregate structured relationships |
| Creative generation | "Draft a risk report based on the current threat landscape" | Moderate — graph provides structured grounding |
GraphRAG consistently outperforms vanilla RAG on tasks where the answer depends on relationships spanning multiple entities, documents, or business rules. The paper identifies three conditions that predict GraphRAG success:
Multi-hop dependency. The answer requires traversing two or more relationship steps (e.g., supplier → component → product → regulation).
Entity centrality. The question revolves around how entities connect rather than what individual documents contain. Questions about compliance exposure, supply chain risk, and organisational impact all benefit.
Hierarchical structure. The data has a natural hierarchy (organisation charts, product categories, regulatory frameworks) that vector similarity cannot capture.
| Benchmark Dimension | GraphRAG | Vanilla RAG | Improvement |
|---|---|---|---|
| Single-hop fact retrieval | 91.2% | 93.1% | −1.9 pp |
| Multi-hop reasoning (2 hops) | 84.7% | 72.3% | +12.4 pp |
| Multi-hop reasoning (3+ hops) | 71.5% | 48.9% | +22.6 pp |
| Contextual summarisation | 88.3% | 76.1% | +12.2 pp |
| Creative generation | 79.8% | 74.2% | +5.6 pp |
The headline finding: GraphRAG underperforms on simple factual lookups but delivers a 12–23 percentage point improvement on multi-hop and relational tasks.
On fact retrieval tasks where the answer is contained in a single document or passage, the overhead of graph construction and traversal is pure cost with no benefit. The paper finds that vanilla vector RAG is faster and equally accurate for:
Based on the GraphRAG-Bench findings, the paper offers three decision rules for enterprise teams:
Use GraphRAG when the question domain is relational by nature — compliance, supply chain, organisational structure, or any regulated workflow where the evidence path must be traceable.
Use vanilla RAG when the question is a simple lookup — document Q&A, policy search, or any task where the answer resides in one or two passages.
Use hybrid GraphRAG (graph + vector) when the task mix is unpredictable — a general-purpose enterprise search that must handle both simple lookups and complex relational queries.
The paper's most valuable contribution for enterprise teams is not a new architecture but a validated decision framework. GraphRAG is not universally superior; it is conditionally superior in precisely defined scenarios. Teams evaluating GraphRAG should start with a workload audit against the paper's three conditions, adopt GraphRAG only for the tasks that satisfy them, and leave simple retrieval on vector search. This targeted approach maximises the ROI of graph infrastructure while avoiding the cost of over-engineering.
The full benchmark, dataset, and evaluation pipeline are available at github.com/GraphRAG-Bench/GraphRAG-Benchmark.