Knowledge Graph Construction from Unstructured Data: A Practical Pipeline

Unstructured text accounts for an estimated 80% of enterprise data — emails, PDFs, internal wikis, support tickets, regulatory filings, and web pages. These documents contain a dense web of entities, relationships, and facts, yet most organisations can only search them by keyword or, at best, by semantic similarity via vector embeddings. Neither approach surfaces the structural connections that make knowledge graphs so powerful: who is connected to what, and how.

This article walks through a production-grade pipeline for transforming unstructured text into a Neo4j knowledge graph, drawing on the latest tools — the Neo4j GraphRAG Python package, LangChain's LLMGraphTransformer, and LLM-based entity resolution — and covering the design decisions that separate a useful graph from a noisy one.

Why Bother? The Limits of Vector-Only RAG

Retrieval-augmented generation (RAG) over vector embeddings retrieves chunks of text that are semantically similar to a user's query. This works well for single-fact questions — "What is the capital of France?" — but fails on multi-hop queries that require reasoning across documents:

"Which products from companies that received FDA clearance in 2025 are relevant to autoimmune diagnostics?"

A vector search returns chunks similar to each fragment of the query, but cannot join across them. A knowledge graph, by contrast, stores entities as nodes and relationships as edges, making multi-hop traversal a single Cypher query:

MATCH (c:Company)-[:HAS_SIGNAL]->(s:Signal {type: "FDA_CLEARANCE"})
WHERE s.date STARTS WITH "2025"
MATCH (c)-[:DEVELOPS]->(a:Application {name: "Autoimmune Diagnostics"})
MATCH (c)-[:SUPPLIES]->(p:Product)
RETURN c.name, p.name, s.date
ORDER BY s.date DESC

This query crosses three relationship types and two entity hops in a single, index-accelerated traversal — something no vector store can express. The graph becomes a structured context layer that an LLM can query with precision.

The Pipeline at a Glance

Building a knowledge graph from unstructured text is a multi-stage pipeline. Each stage introduces a design choice that affects the quality and utility of the final graph:

Raw Documents
    ↓
[1] Data Loading & Chunking
    ↓
[2] Embedding & Lexical Graph Construction
    ↓
[3] Schema-Guided Entity & Relation Extraction
    ↓
[4] Knowledge Graph Writing
    ↓
[5] Entity Resolution & Deduplication
    ↓
[6] Post-Processing & Enrichment
    ↓
Queryable Knowledge Graph

Stage 1: Data Loading and Chunking

The pipeline begins with ingesting documents from whatever source they live in — PDFs, web pages, S3 buckets, YouTube transcripts, or plain text files.

The neo4j-graphrag Python package provides a SimpleKGPipeline that abstracts most of this away:

from neo4j import GraphDatabase
from neo4j_graphrag.llm import OpenAILLM
from neo4j_graphrag.embeddings import OpenAIEmbeddings
from neo4j_graphrag.pipeline import SimpleKGPipeline

driver = GraphDatabase.driver("neo4j+s://your-instance.databases.neo4j.io",
                              auth=("neo4j", "password"))
llm = OpenAILLM(model_name="gpt-4o", model_params={"temperature": 0})
embedder = OpenAIEmbeddings(model="text-embedding-3-small")

kg_builder = SimpleKGPipeline(
    llm=llm,
    driver=driver,
    embedder=embedder,
    from_pdf=True,
)

kg_builder.run("path/to/document.pdf")

Chunking strategy is the first critical design decision. The Neo4j pipeline uses a token-based splitter that respects the LLM's context window. Key parameters:

Parameter	Effect
Small chunks (256–512 tokens)	More precise entity extraction, but loses cross-sentence context
Large chunks (1024–2048 tokens)	Better relationship detection across sentences, but risks exceeding context windows
Chunk overlap (10–20%)	Prevents entity mentions from being split across chunk boundaries

Rule of thumb: match chunk size to your LLM's context window, and always include overlap. A 10–20% overlap ensures that entities spanning chunk boundaries are still captured in at least one extraction pass.

Stage 2: Lexical Graph Construction

Before extracting domain entities, the pipeline builds a lexical graph — a structural representation of the document itself:

(:Document {path: "report.pdf"})
    -[:FROM_DOCUMENT]->
(:Chunk {text: "...", embedding: [...]})
    -[:NEXT_CHUNK]->
(:Chunk {text: "...", embedding: [...]})

The lexical graph serves two purposes. First, it preserves provenance: every extracted entity is linked back to the chunk it came from, and every chunk is linked to its source document. Second, the chunk embeddings enable hybrid retrieval — vector search for relevance, graph traversal for context — which is the foundation of GraphRAG.

The neo4j-graphrag package builds the lexical graph automatically when SimpleKGPipeline runs. The FROM_DOCUMENT and NEXT_CHUNK relationships are created by the LexicalGraphBuilder component, which can be configured or replaced if custom chunking logic is required.

Stage 3: Schema-Guided Entity and Relation Extraction

This is where the real intelligence lives. An LLM reads each chunk and extracts entities and relationships according to a schema — a definition of what node types, relationship types, and properties the graph should contain.

Why Schema Matters

Without a schema, an LLM might produce labels like Person, Company, Organisation, and Organization for the same conceptual type, or create relationships with inconsistent directionality. A schema grounds the extraction and ensures consistency across all chunks.

kg_builder = SimpleKGPipeline(
    llm=llm,
    driver=driver,
    embedder=embedder,
    schema={
        "node_types": [
            {"label": "Company", "properties": ["name", "domain", "region"]},
            {"label": "Product", "properties": ["name", "category"]},
            {"label": "Application", "properties": ["name"]},
        ],
        "relationship_types": [
            {"label": "SUPPLIES", "source": "Company", "target": "Product"},
            {"label": "DEVELOPS", "source": "Company", "target": "Application"},
            {"label": "COMPETES_WITH", "source": "Company", "target": "Company"},
        ],
    }
)

As of neo4j-graphrag v1.8.0, the schema can also be automatically inferred from the input text using the SchemaFromTextExtractor component, which analyses a sample of the text and proposes node and relationship types. You can then review, edit, and save the inferred schema before running the full extraction:

from neo4j_graphrag.experimental.pipeline.kg_builder import SchemaFromTextExtractor

extractor = SchemaFromTextExtractor(llm=llm)
inferred_schema = extractor.run(text=sample_text)
inferred_schema.save("inferred_schema.json")

The automatic schema extraction is particularly valuable when working with unfamiliar domains where you cannot predict the entity types in advance.

Structured Output for Reliability

When using OpenAI or Vertex AI as the LLM backend, the pipeline supports structured output mode. Instead of relying on prompt engineering to produce valid JSON, the LLM returns typed Pydantic models directly, which eliminates parsing errors:

kg_builder = SimpleKGPipeline(
    llm=OpenAILLM(
        model_name="gpt-4o",
        supports_structured_output=True,  # enables response_format
    ),
    ...
)

In our testing, structured output reduced extraction errors — malformed JSON, hallucinated property names, inconsistent label casing — by roughly 70% compared to prompt-only extraction.

Stage 4: Entity Resolution

Because each chunk is processed independently, the same entity mentioned in different chunks produces duplicate nodes. "Neo4j Inc.", "neo4j", and "Neo4j" might all end up as separate :Company nodes.

The EntityResolver component handles this with configurable strategies:

Strategy	Mechanism	Best For
Exact match	Merges nodes with identical label and name property	Clean, consistent input
Fuzzy match	Levenshtein distance via RapidFuzz	OCR text, minor spelling variations
Semantic match	spaCy embedding cosine similarity	Synonymous names ("Neo4j" vs "Neo4j Inc.")
LLM-based	LLM decides whether two entities refer to the same real-world object	Complex cases with context-dependent identity

from neo4j_graphrag.experimental.pipeline.kg_builder import (
    EntityResolver,
    FuzzyMatchResolver,
)

resolver = EntityResolver(
    resolvers=[
        FuzzyMatchResolver(similarity_threshold=0.85),
    ]
)
await resolver.run(driver=neo4j_driver, database="neo4j")

The resolution step is the difference between a graph with 15,000 noisy entities and one with 3,000 clean, well-merged nodes. Invest in this stage.

Stage 5: Post-Processing and Enrichment

Once entities are resolved, the graph can be enriched:

1. kNN similarity links — clusters of similar chunks are connected with SIMILAR relationships, enabling graph traversal during retrieval
2. Community detection — the Neo4j Graph Data Science library's Louvain or Weakly Connected Components algorithms identify entity clusters, which can be summarised by an LLM to create community-level context for GraphRAG
3. Full-text and vector indexes — enable hybrid search across both semantic similarity and keyword matching

// Create a vector index for chunk similarity search
CREATE VECTOR INDEX chunks IF NOT EXISTS
FOR (c:Chunk) ON (c.embedding)
OPTIONS {indexConfig: {
  `vector.dimensions`: 1536,
  `vector.similarity_function`: 'cosine'
}};

Extraction Approaches Compared

Approach	Setup Effort	Accuracy	Consistency	Scalability	Cost
Regex / rules	High	Low-medium	High	Excellent	Free
NER model (spaCy, Stanza)	Medium	Medium	High	Good	Free–low
LLM prompt-based	Low	High	Medium	Moderate	Per-token
LLM structured output	Low	High	High	Moderate	Per-token
Hybrid (NER + LLM)	Medium	Very high	High	Good	Low–medium

For production systems, the hybrid approach is increasingly common: a lightweight NER model (spaCy or GLiNER) pre-filters candidate entities, and an LLM validates and enriches them with relationship context. This reduces LLM token consumption by 60–80% while maintaining extraction quality.

Putting It All Together: A Production Pipeline

Here is a minimal but complete pipeline that ingests a PDF and produces a queryable knowledge graph:

import os
from neo4j import GraphDatabase
from neo4j_graphrag.llm import OpenAILLM
from neo4j_graphrag.embeddings import OpenAIEmbeddings
from neo4j_graphrag.pipeline import SimpleKGPipeline

# 1. Connect to Neo4j
driver = GraphDatabase.driver(
    os.environ["NEO4J_URI"],
    auth=(os.environ["NEO4J_USER"], os.environ["NEO4J_PASSWORD"]),
)

# 2. Configure LLM and embedder
llm = OpenAILLM(
    model_name="gpt-4o-mini",
    supports_structured_output=True,
)
embedder = OpenAIEmbeddings(model="text-embedding-3-small")

# 3. Define schema
schema = {
    "node_types": [
        {"label": "Person", "properties": ["name", "role"]},
        {"label": "Organization", "properties": ["name", "industry"]},
        {"label": "Technology", "properties": ["name", "category"]},
    ],
    "relationship_types": [
        {"label": "WORKS_AT", "source": "Person", "target": "Organization"},
        {"label": "DEVELOPS", "source": "Organization", "target": "Technology"},
        {"label": "PARTNERS_WITH", "source": "Organization", "target": "Organization"},
    ],
}

# 4. Build and run pipeline
kg_builder = SimpleKGPipeline(
    llm=llm,
    driver=driver,
    embedder=embedder,
    schema=schema,
    from_pdf=True,
)

kg_builder.run("market_report.pdf")

After the pipeline completes, you can query the graph directly:

-- Find all technologies developed by partners of a given company
MATCH (org:Organization {name: "Siemens Healthineers"})-[:PARTNERS_WITH]->(partner)
MATCH (partner)-[:DEVELOPS]->(tech:Technology)
RETURN partner.name AS Partner, tech.name AS Technology;

Conclusion

Knowledge graph construction from unstructured data has moved from a bespoke, labor-intensive process to a repeatable pipeline, thanks to LLMs and the Neo4j graph ecosystem. The key design decisions are:

1. Chunk with overlap — token boundaries should never split an entity mention
2. Use a schema — without one, label inconsistency will plague your graph
3. Resolve entities — deduplication is not optional; it is the difference between a graph and a tangle
4. Preserve provenance — always link extracted entities back to source chunks
5. Leverage structured output — it eliminates the most common failure mode of LLM-based extraction

The tools are mature enough — the neo4j-graphrag package, SimpleKGPipeline, SchemaFromTextExtractor, and EntityResolver — that a single developer can go from raw PDFs to a queryable knowledge graph in an afternoon. The bottleneck is no longer technology; it is ontology design. Knowing what to extract and how to model it remains the irreplaceable human skill.

Further Reading

• Neo4j GraphRAG Python Package Documentation — Official reference for the pipeline components used in this article
• Knowledge Graphs as a Context Layer for AI Agents — How to connect constructed graphs to agent workflows
• Ontology in Graph Databases — Schema and ontology design principles for production graphs
• Introduction to GraphRAG — Using the constructed graph for retrieval-augmented generation
• Neo4j. Constructing Knowledge Graphs with Neo4j GraphRAG for Python. GraphAcademy Course — Hands-on tutorial covering custom loaders, schema design, and GraphRAG retrievers