Do You Even Need the Graph? An Honest Yield Test Before You Build GraphRAG

Every knowledge-graph vendor has a slide that begins "you don't need just a vector store." It's usually part of a larger pitch. But a surprising amount of "GraphRAG" is a cool solution in search of a question it genuinely needs — and the construction, validation, and orchestration cost is real. The professional move before building the graph is to run a yield test: prove that your questions actually require joined relationships a vector store can't deliver.

The honest baseline: what vector search already gives you

Vector search answers "which chunk is most similar to this question". For a large class of real questions that is enough, or enough most of the time:

• Single-fact lookup — "what is the VLAN of the finance network?"
• Summarization of related chunks — "summarize the incident runbook".
• Browse-by-topic — "what do we know about Qwen models?"

If the overwhelming majority of your top questions are in these buckets, a vector store already handles them. A graph adds latency, maintenance, and a second system for zero answer gain on those questions.

The yield test: find the questions that demand joins

Graphs earn their keep only when an answer requires combining facts across entities in a relationship structure — where the answer is not in any single chunk but emerges from how things connect. Concrete shapes:

• Multi-hop dependency — "every service that depends, directly or transitively, on the MongoDB that Tobi's team owns".
• Shared-neighbor reasoning — "the authors who co-edit with Tobi beyond the current project".
• Path/count aggregation over typed edges — "vendors appearing in more than one department's stack".

The test: take your real top-N questions and label each one vector-satisfiable or graph-required. If graph-required is a small minority, you have not found your graph use-case yet — no amount of modeling will change that.

A concrete threshold to argue with

Use the labeling to drive a decision with numbers, not vibes:

score = (questions where a graph changes the answer) / (all questions)

score >  20%   → build the graph; it carries its weight
score ≈ 5–20% → build a *targeted* graph covering only that slice,
                keep vector for the rest (hybrid)
score <  5%    → the graph is theater; fix retrieval or accept vector-only

The targeted/hybrid middle band is where the honesty pays off: not "do we build a graph" but "do we build a 200-node graph over exactly the relationship-heavy slice." That is dramatically cheaper than a "knowledge graph strategy" and answers the same questions.

Guard against the trap questions

Two kinds of trap inflate your graph-required count:

• Phrased to need a graph. "Give me dependency relationships" is a retrieval request — vector can surface it. The question must be one where the answer requires joins, not one that merely mentions relationships.
• The demo question. "Which two things connect our people and our vendors?" sounds impressive and is almost never a real user question. Test against the questions users actually ask, not the ones in the keynote.

Run the test before you build, and re-run it as you scale

Run the yield test before any modeling. Mark the scores. If you later find the workload drifting relationship-heavy, re-run the labeling — a growing graph-required share is the legitimate signal to invest. The graph is a tool for a specific class of question, and the cheapest way to know whether you're in that class is to test the questions first instead of building the infrastructure on faith.

Knowing when not to build a graph is as valuable as knowing how to build one. The yield test keeps "GraphRAG" honest — and on the days it says "you don't need the graph", it has just saved you the most expensive thing in the project: the months you would have spent building it anyway.