Graph Construction from Text with LLMs: The Bottleneck Hiding in Your GraphRAG Stack

GraphRAG post-mortems almost always blame retrieval. They are looking in the wrong place. The accuracy ceiling in a graph-augmented system is set upstream, the moment raw text is turned into nodes and edges — a step most teams treat as a pip install away from done. The research corpus agrees: graph construction — extracting the graph from unstructured text — produced 187 papers in the first eight months of 2026 alone, more than triple its prior-year pace, and it is now the fastest-moving sub-field feeding GraphRAG.

A weak graph cannot be rescued by a clever retriever. If "Dr. GraphWiz AI" and "GraphWiz" are stored as two nodes, no traversal will ever connect them; if a causal relation is silently dropped, no prompt will recover it. This article walks a production-grade construction pipeline you can run today, and shows where the precision is actually won and lost.

The pipeline in five stages

Construction is not a single model call. It is a pipeline with distinct failure modes:

1. Chunking — bound context and preserve locality.
2. Schema definition — fix the entity and relation vocabulary.
3. Extraction — emit typed nodes and edges per chunk.
4. Entity resolution — merge "GraphWiz" and "GraphWiz AI" into one node.
5. Loading & validation — write to Neo4j and verify.

Stages 3 and 4 carry the most risk. Get them right and the rest is plumbing.

1. Chunking with overlap

Naïve fixed-size splits sever relations that span a boundary. Use recursive character splitting with a hard overlap so a relation mentioned across a seam is captured in at least one chunk.

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1500,
    chunk_overlap=200,      # keeps cross-boundary relations intact
    separators=["\n\n", "\n", ". ", " "],
)
chunks = splitter.split_text(document)

2. Schema: fixed beats discovered (at first)

Letting the model invent types per chunk produces a graph no query can navigate. Start with a fixed schema — a closed vocabulary of entity and relation types — and only graduate to LLM-discovered schema once you have evaluated the fixed one.

SCHEMA = """
ENTITY TYPES: PERSON, ORG, PRODUCT, CONCEPT, TECHNOLOGY
RELATION TYPES: WORKS_AT, MENTIONS, CAUSES, PART_OF, BUILT_WITH
"""

3. Extraction with structured output

Structured output turns the model into a deterministic extractor. Pydantic models give you validation for free, and temperature=0 keeps extractions reproducible.

from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

class Entity(BaseModel):
    name: str = Field(description="Canonical entity name")
    type: str = Field(description="One of PERSON, ORG, PRODUCT, CONCEPT, TECHNOLOGY")

class Relation(BaseModel):
    source: str
    target: str
    label: str = Field(description="One of WORKS_AT, MENTIONS, CAUSES, PART_OF, BUILT_WITH")

class Extraction(BaseModel):
    entities: list[Entity]
    relations: list[Relation]

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chain = (
    ChatPromptTemplate.from_messages([
        ("system", "Extract entities and typed relations. Use ONLY the provided schema."),
        ("human", "SCHEMA:\n{schema}\n\nTEXT:\n{text}"),
    ])
    | llm.with_structured_output(Extraction)
)

result = chain.invoke({"schema": SCHEMA, "text": chunk})

4. Entity resolution — the step that decides quality

Extraction gives you mentions; resolution gives you entities. Without it, every rephrasing of a name becomes a new node and the graph fragments. The cheap, effective pattern is blocking then canonicalisation: group candidates by type and initial, then ask the model to pick one canonical name per block.

def resolve(mentions: list[Entity], llm) -> dict[str, str]:
    # Blocking: avoid O(n^2) pairwise comparison
    blocks: dict[tuple, list[Entity]] = {}
    for m in mentions:
        blocks.setdefault((m.type, m.name[0].lower()), []).append(m)

    mapping = {}
    for block in blocks.values():
        names = [m.name for m in block]
        if len({n.lower() for n in names}) == 1:
            canonical = names[0]
        else:
            canonical = llm.invoke(
                f"These may refer to one entity: {names}. "
                f"Return the single best canonical name, or 'DISTINCT' if they differ."
            ).content
        for m in block:
            mapping[m.name] = canonical if canonical != "DISTINCT" else m.name
    return mapping

For high-stakes graphs, agentic resolution (grounded neuro-symbolic frameworks such as ANCHOR-RE, published in 2026) outperforms flat canonicalisation by reasoning over evidence before merging. Adopt it only where a wrong merge is costly — it multiplies latency and token spend.

5. Loading and validating in Neo4j

Write through parameterised Cypher so every mention resolves to its canonical node and duplicate edges collapse into one relationship.

MERGE (e:Entity {id: $canonical})
SET e.type = $type
WITH e
UNWIND $relations AS r
MATCH (s:Entity {id: r.source}), (t:Entity {id: r.target})
MERGE (s)-[:REL {type: r.label}]->(t)

Run a post-load assertion: MATCH (n:Entity) RETURN count(n) should grow sub-linearly as documents accumulate — if node count tracks document count, resolution is failing.

Tooling landscape

You rarely start from scratch. The mature options differ sharply in where they spend effort.

Tool	Construction style	Resolution	Best for
LlamaIndex KnowledgeGraphIndex	Single-shot LLM extract	Weak (none by default)	Prototypes, small corpora
Microsoft GraphRAG	Claim + community summarisation	Entity merge built in	Broad corpora needing summaries
Neo4j llm-graph-builder	UI-driven pipelines	Configurable	Ops teams, Neo4j shops
Custom (above)	Full control	Yours to design	Precision-critical, evolving schema

The differentiator is almost never the extractor — it is resolution and evaluation, which the off-the-shelf tools handle inconsistently.

Evaluation you can actually trust

Construction has no compiler. The only way to know your graph is faithful is to sample and judge. Take 50 extracted triples, present each beside its source sentence, and let a separate model rule on faithfulness.

def eval_triple(triple: str, sentence: str, judge) -> bool:
    verdict = judge.invoke(
        f"Is the relation '{triple}' faithfully supported by this sentence?\n"
        f"SENTENCE: {sentence}\nAnswer 'yes' or 'no' with a one-line reason."
    ).content.lower()
    return verdict.startswith("yes")

sample = random.sample(extracted_triples, 50)
precision = sum(eval_triple(t.relation, t.source_sentence, judge) for t in sample) / len(sample)

Track extraction precision (is each triple grounded?) and resolution recall (are true duplicates merged?) as first-class metrics, exactly as you would track retrieval recall downstream. A graph scoring below ~0.85 precision will quietly degrade every later query.

Trade-offs and when to go agentic

Choice	Cost	Benefit
Fixed schema	Low	Navigable, queryable graph
Discovered schema	Medium	Adapts to new corpora
Agentic resolution	High (latency + tokens)	Fewer wrong merges

Start with a fixed schema and Pydantic extraction, add resolution before the first load, and evaluate on a 50-triple sample with an LLM judge. Only reach for agentic extraction (ANCHOR-RE-style) where a wrong merge is expensive — fraud, safety, or regulatory graphs.

Next steps: stand up the five-stage pipeline against one dense document this afternoon; assert node growth is sub-linear after resolution; and wire extraction precision into the same dashboard that already tracks your retriever. The retriever was never the only lever — now you have the other one.