GraphRAG#
GraphRAG is a retrieval pattern that augments an LLM with a knowledge graph instead of (or in addition to) a flat vector index. It is the answer to a specific weakness of vector RAG: questions that need multi-hop reasoning or a global view of the corpus.
flowchart LR
Q([Query]) --> Router{Local or global}
Router -->|local entity| GT[Graph traversal]
Router -->|global theme| CS[Community summaries]
KG[(Knowledge graph)] --> GT
KG --> CS
GT --> LLM[Generator LLM]
CS --> LLM
LLM --> A([Grounded answer])
classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
class Q,A client;
class Router,GT,CS,LLM service;
class KG datastore;
The limit of vector RAG#
Plain vector RAG embeds chunks and retrieves by cosine similarity. It is excellent at one job: find the passage that lexically or semantically matches the question. It struggles with two query shapes:
- Multi-hop: "Which engineers worked on services owned by teams that report to the VP of Platform?" There is no single chunk that contains this answer. The information lives in three documents that share no vocabulary.
- Global / aggregate: "What are the main themes in this 10,000-page archive?" There is no top-K chunk that summarises the whole corpus. Every chunk is local by definition.
GraphRAG fixes both by representing the corpus as a graph of typed entities and labelled relationships, then traversing or summarising the graph at query time.
What a knowledge graph gives you#
A knowledge graph stores facts as (subject, predicate, object) triples or as labelled nodes and edges. Three things become trivial that were hard before:
- Traversal: follow an entity through two or three relationships to surface connected facts.
- Aggregation: count, group, or filter over edges, not just match strings.
- Topic clustering: communities of densely connected entities map to coherent themes.
The graph is not a replacement for the vector index. The two layers work together: the vector index finds the right document, the graph explains how documents connect.
The GraphRAG pipeline#
The pipeline has five stages, mostly offline:
- Entity and relationship extraction: an LLM reads each chunk and emits typed entities (
Person,Service,Team) plus typed relationships (OWNS,REPORTS_TO,MODIFIED). - Graph build: dedupe entities by normalised name and embedding, then write nodes and edges into a graph database. Neo4j, Memgraph, or an in-memory NetworkX graph all work.
- Community detection: run Leiden or Louvain on the graph to find densely connected clusters. Each cluster is a candidate topic.
- Per-community summaries: prompt an LLM with the entities and edges of each community to generate a short summary. Store these alongside the graph.
- Query routing: at query time, classify the question as local (anchored on a specific entity) or global (cross-cutting). Local queries get entity linking plus graph traversal; global queries get a map-reduce over community summaries.
Microsoft GraphRAG: the contribution#
Microsoft Research published GraphRAG in early 2024 and open-sourced the pipeline. The main contribution is not the entity-extraction step (that is older work); it is the community-summary trick that makes global queries cheap at query time. Pre-generating summaries during indexing means a "what are the themes" question becomes a map-reduce over a few hundred summaries instead of a full corpus scan.
The Microsoft paper reports large quality gains over naive vector RAG on synthesis and "sense-making" benchmarks, especially on questions like "what is this dataset about" that pure retrieval cannot answer.
When GraphRAG wins, when it does not#
GraphRAG wins clearly when:
- Questions require linking facts across documents.
- The corpus has a strong entity skeleton: people, services, products, papers.
- Users ask synthesis questions, not just lookups.
GraphRAG is the wrong tool when:
- The corpus is small and questions are local. Vector RAG plus reranking is cheaper.
- The corpus changes minute to minute. The extraction cost dominates.
- Entities are weak (free-form prose with no recurring nouns).
In production, GraphRAG usually sits next to a vector index, not instead of it. The router picks the layer to consult per query, and falls back to the other when the first comes back empty.
A concrete example#
Imagine a query against an engineering knowledge base: "What does our codebase say about how user X interacts with feature Y?" A vector retriever would fetch chunks that mention user X or feature Y, often missing the connecting tissue. A graph traversal walks from User(X) to events to feature flags to the services that implement feature Y, then pulls the source chunks attached to those edges. The graph wins because the answer is a path, not a passage.
The cost is extraction: indexing 10,000 docs with an LLM call per chunk is expensive and slow. The win is that once the graph exists, traversal at query time is essentially free.
GraphRAG is a retrieval pattern that complements vector search with an LLM-extracted knowledge graph. It targets two query shapes that flat embedding retrieval handles badly: multi-hop reasoning across documents, and global synthesis across a whole corpus. This guide walks the full pipeline, the Microsoft GraphRAG paper's specific contribution, a Cypher traversal example, and when the cost of graph construction is worth paying.
Why vector RAG hits a wall#
Vector RAG retrieves chunks by approximate nearest neighbour search in an embedding space. The pattern is fast, simple, and surprisingly strong on lookup-style questions. It breaks on two query families.
Multi-hop questions need the system to combine facts that live in different documents and share little vocabulary. A query like "Which engineer reviewed the most pull requests for services owned by the platform team last quarter?" has no chunk that contains the full answer. The system must connect Engineer to PullRequest, PullRequest to Service, and Service to Team, then aggregate. Cosine similarity cannot do that.
Global / aggregate questions ask about the corpus as a whole. "What are the main themes in this 10,000-document archive?" or "Which research areas overlap between these two labs?" There is no top-K of chunks that summarises the corpus, because summary is not a lexical property of any single chunk.
GraphRAG attacks both by exposing the corpus as a structured graph rather than a bag of vectors. Microsoft's 2024 GraphRAG paper made this concrete, open-sourced the pipeline, and showed double-digit improvements on synthesis benchmarks over naive RAG.
What a knowledge graph adds#
A knowledge graph stores typed entities (nodes) and typed relationships (edges). The same facts can sit in a vector index as text and in a graph as structure. The graph adds three capabilities the vector index lacks:
- Multi-hop traversal: follow edges from one entity to discover connected facts.
- Aggregation over relationships: count, group, filter by edge type.
- Cluster detection: find dense subgraphs that correspond to topics.
GraphRAG does not throw away the vector index. Most production pipelines run both: the vector index for local lookups, the graph for traversal and global queries. A router picks the right layer (or both) for each question.
End-to-end architecture#
flowchart TB
subgraph Index[Indexing pipeline, offline]
D[Docs] --> Ch[Chunker]
Ch --> EX[LLM extractor]
EX --> DD[Dedupe and normalise]
DD --> KG[(Graph DB)]
KG --> CD[Community detection]
CD --> CS[Community summarizer LLM]
CS --> SS[(Summary store)]
Ch --> Emb[Embedder]
Emb --> VI[(Vector index)]
end
subgraph Query[Query pipeline, online]
Q[User query] --> R[Router LLM]
R -->|local| EL[Entity linker]
EL --> GT[Graph traversal]
KG --> GT
R -->|global| MR[Map-reduce over summaries]
SS --> MR
R -->|fallback| VR[Vector retrieval]
VI --> VR
GT --> Ctx[Context builder]
MR --> Ctx
VR --> Ctx
Ctx --> Gen[Generator LLM]
Gen --> A[Answer]
end
classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
class Q,A client;
class R,EL,GT,MR,VR,Ctx,Gen service;
class Ch,EX,DD,CD,CS,Emb compute;
class KG,SS,VI datastore;
The offline path dominates the cost: one LLM call per chunk for extraction, one per community for summarisation. The online path is cheap: graph traversal is microseconds, summary map-reduce is a few LLM calls.
Stage 1: entity and relationship extraction#
For each chunk, the extractor LLM is prompted to emit a structured list of entities and relationships. A typical schema:
{
"entities": [
{"id": "alice", "type": "Person", "name": "Alice Chen"},
{"id": "billing", "type": "Service", "name": "Billing"},
{"id": "platform", "type": "Team", "name": "Platform"}
],
"relationships": [
{"source": "alice", "target": "billing", "type": "MODIFIED", "evidence": "PR #4821"},
{"source": "platform", "target": "billing", "type": "OWNS", "evidence": "OWNERS file"}
]
}
Three things matter for quality:
- Schema: a fixed list of entity types and relationship types beats free-form extraction. The LLM gets a typed prompt and produces fewer hallucinations.
- Evidence: every relationship carries a pointer back to the source chunk. This is what lets the generator cite passages, not just facts.
- Embeddings on entities: each entity gets an embedding from its name plus a description. The embedding is used for dedup and for entity linking at query time.
Microsoft's GraphRAG uses gpt-4-class models for this step. Cheaper models work for narrower schemas; the quality gap matters more than the cost gap below ~1M chunks.
Stage 2: dedupe and graph build#
Raw extraction produces duplicate entities: "Alice", "Alice Chen", and "A. Chen" become three nodes when they should be one. The dedup pass groups entities by normalised name and embedding similarity, then merges them. The resulting graph is written to a graph database.
flowchart LR
E1[Raw entity: Alice] --> M{Merge?}
E2[Raw entity: Alice Chen] --> M
E3[Raw entity: A. Chen] --> M
M -->|name + embedding match| OneNode[(Single Person node)]
classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
class M service;
class OneNode datastore;
Neo4j is the common production choice; Memgraph is faster for read-heavy workloads; for prototypes an in-memory NetworkX graph is fine. The choice does not change the pipeline shape.
Stage 3: community detection#
The graph is now a tangled mess of nodes and edges. To answer global queries cheaply, GraphRAG runs a community-detection algorithm (Leiden by default, Louvain as the older alternative) to find densely connected clusters. Each community is a candidate topic.
flowchart TB
subgraph G[Knowledge graph]
A1((A)) --- A2((B))
A2 --- A3((C))
A1 --- A3
B1((D)) --- B2((E))
B2 --- B3((F))
A3 -.- B1
end
G --> L[Leiden]
L --> C1[Community 1: A B C]
L --> C2[Community 2: D E F]
classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
class L service;
Leiden produces hierarchical communities at multiple resolutions. Higher resolutions yield small, tight clusters; lower resolutions yield broad themes. GraphRAG stores summaries at several levels so the router can pick the right granularity.
Stage 4: per-community summaries#
For each community, an LLM is prompted with the community's entities, edges, and a sample of evidence chunks to produce a short paragraph summary. This is the trick that makes global queries fast: at query time, "what are the themes in this corpus" is a map-reduce over a few hundred community summaries, not a full scan.
The summaries are stored alongside the graph, indexed by community ID and embedded for retrieval.
Stage 5: query routing#
At query time, a small router LLM (or a classifier) decides whether the question is local or global.
- Local query: anchored on a specific entity. The system links query mentions to graph nodes, then runs a bounded traversal (1 to 3 hops) and pulls the evidence chunks attached to the visited edges.
- Global query: about the corpus as a whole. The system retrieves the top-K community summaries by relevance to the question, then runs a map-reduce: map a per-community partial answer, reduce into a final answer.
- Fallback: if either path returns thin context, the router falls back to vector RAG.
A Cypher traversal example#
Concretely, a local GraphRAG query against a Neo4j-backed graph might look like this. The query asks "Which engineers modified billing services owned by the Platform team in the last 90 days, and what did they change?"
MATCH (p:Person)-[m:MODIFIED]->(s:Service)<-[:OWNS]-(t:Team {name: 'Platform'})
WHERE s.domain = 'billing'
AND m.timestamp > datetime() - duration({days: 90})
RETURN p.name AS engineer,
s.name AS service,
m.evidence AS source_chunk,
m.summary AS change_summary
ORDER BY m.timestamp DESC
LIMIT 25
The traversal walks three relationships (Team OWNS Service, Person MODIFIED Service, plus the timestamp filter) in one query. A vector index could not answer this without manual orchestration. The m.evidence column is what gets handed to the generator LLM so the final answer cites the actual source passages, not just the graph facts.
Equivalent SPARQL is straightforward if the graph is RDF rather than property-graph; the shape of the traversal is the same.
Microsoft GraphRAG: what the paper actually adds#
The 2024 Microsoft Research paper "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" is what made GraphRAG a recognised pattern. The contributions, ranked by what matters in interviews:
- Community summaries as a first-class query target: pre-generating summaries during indexing turns expensive corpus-wide questions into a cheap map-reduce. This is the load-bearing idea.
- Hierarchical Leiden communities: multi-resolution clusters give the router a knob to pick granularity per query.
- Open-source reference pipeline: a working implementation in Python plus prompts plus eval harness. The prompts in particular are reused widely.
- Evaluation on sense-making benchmarks: large gains over naive RAG on questions where the ground truth is a synthesis rather than a lookup.
The entity-extraction step is older work (OpenIE, REBEL, and friends). What was new was the routing pattern and the community trick.
Trade-offs#
| Dimension | Vector RAG | GraphRAG |
|---|---|---|
| Index cost | One embedding call per chunk | One LLM extraction call per chunk + community summaries |
| Index time on 1M docs | Hours | Days |
| Query latency | 50 to 200 ms | 200 ms to 1 s including LLM map-reduce |
| Multi-hop quality | Poor | Strong |
| Global synthesis quality | Very poor | Strong |
| Freshness story | Easy: re-embed changed docs | Hard: re-extract, dedupe, possibly re-summarise communities |
| Maintenance | Just the index | Schema, extractor prompts, dedup thresholds, summary refresh |
The graph also drifts. Extraction is noisy: a model upgrade or a schema change can shift the entity space and force re-indexing. Mature pipelines version the extractor and shadow-index against the old version during a rollout.
When to use GraphRAG vs vector RAG vs hybrid#
- Plain vector RAG: short corpus, lookup-style questions, fast iteration. The default.
- Hybrid (vector + BM25 + reranker): jargon-heavy or acronym-heavy corpora. The production workhorse.
- GraphRAG only: small entity-dense corpora where queries are almost all multi-hop or synthesis. Rare.
- GraphRAG layered on hybrid: the usual production answer when you need both lookup and synthesis. The router decides per query.
A useful heuristic: if you cannot name 10 entity types up front that cover your corpus, GraphRAG is probably not worth the extraction cost. If you can name them and they recur across documents, the graph will pay back.
A concrete example query#
Imagine an internal codebase question: "What does our codebase say about how user kshitij_s interacts with feature recommendations_v2?" A vector retriever returns chunks containing either string, often without the connecting tissue. A graph traversal goes:
- Resolve
kshitij_sto aUsernode. - Walk
User -[INTERACTED_WITH]-> Event -[TRIGGERED]-> FeatureFlag -[CONTROLS]-> Service. - Filter
Servicenodes whosefeatureproperty equalsrecommendations_v2. - Pull the evidence chunks attached to each visited edge.
The generator now has a coherent path plus source passages and can write a structured answer. The vector retriever, asked the same question, returns the right chunks individually but loses the relationship between them. GraphRAG wins because the answer is a path, not a passage.
Production failure modes#
- Extraction noise: typos in entity names, hallucinated relationships. Mitigate with strict schemas, evidence checks, and a confidence threshold on edges.
- Stale graph: docs change, entities move. Use document hashes to drive incremental re-extraction; re-summarise only communities whose membership changed.
- Schema drift: a new entity type appears, breaks downstream consumers. Treat the schema as code, version it, gate changes.
- Summary staleness: community summaries lag the underlying graph. Stamp summaries with the version of the graph they were generated against; recompute when divergence crosses a threshold.
- Router miscalibration: local queries routed to summaries get vague answers; global queries routed to traversal time out. Run an offline benchmark per router release.
Evaluation#
The same three numbers that matter for vector RAG matter here, plus two graph-specific ones:
- Recall@K on local queries: did the right chunks make it into the context?
- End-to-end answer quality: LLM-as-judge or human eval, especially on multi-hop questions.
- Synthesis quality on global queries: human eval, since there is no ground-truth chunk.
- Entity precision and recall: against a hand-labelled gold set of entities and relationships.
- Community coherence: do humans agree the per-community summaries describe coherent topics?
The graph-specific numbers are what let you debug regressions: a drop in answer quality is often an entity-precision regression upstream.
FAQ#
What is GraphRAG?#
GraphRAG is a retrieval pattern that builds a knowledge graph of entities and relationships from a corpus using an LLM, then traverses that graph at query time. It complements vector RAG by handling multi-hop and global questions that pure embedding similarity cannot.
How is GraphRAG different from vector RAG?#
Vector RAG retrieves chunks by cosine similarity, which works for local lookups but misses connections across documents. GraphRAG stores entities and edges explicitly and answers questions that require following two or more relationships or summarising a whole topic cluster.
What is Microsoft GraphRAG?#
Microsoft GraphRAG is the 2024 open-source pipeline that extracts entities and relationships with an LLM, runs Leiden community detection on the resulting graph, and pre-generates per-community summaries to answer global queries like what are the main themes in this corpus.
When should you use GraphRAG instead of vector RAG?#
Use GraphRAG when questions cross many documents, depend on entity relationships, or ask for synthesis across a topic. Stick with vector RAG when queries are local lookups against clean docs and you cannot afford the extraction cost.
How does GraphRAG construct its knowledge graph?#
An LLM reads each text chunk and emits typed entities and typed relationships in a structured schema. A post-processor deduplicates entities by normalised name and embedding similarity, then writes the result to a graph database such as Neo4j or Memgraph.
What is community detection in GraphRAG?#
Community detection is a graph algorithm such as Leiden or Louvain that finds densely connected clusters of nodes. GraphRAG runs it on the extracted entity graph and treats each community as a candidate topic, then generates a short LLM summary per community for use at query time.
What is the difference between a local and a global GraphRAG query?#
A local query anchors on one or two known entities and answers by traversing edges out from them. A global query is corpus wide, such as what are the main themes, and is answered by map-reducing over the pre-generated community summaries instead of touching the graph directly.
How does GraphRAG handle stale data?#
Incrementally. Changed documents are re-extracted, new entities and edges are upserted, and any community whose membership changed is re-summarised. Full rebuilds are reserved for schema changes or extractor model upgrades.
Is GraphRAG always better than vector RAG?#
No. GraphRAG wins on multi-hop and global questions but is far more expensive to build and harder to keep fresh. Most production systems run both layers and route per query rather than picking one.
What is the GraphRAG pipeline in one sentence?#
Extract entities and relationships from chunks with an LLM, build a graph, run community detection, summarise each community, then route queries to either graph traversal or summary map-reduce.
What does Microsoft GraphRAG add over older knowledge-graph RAG work?#
Pre-generated per-community summaries that turn corpus-wide questions into a cheap map-reduce, plus a working open-source reference pipeline with prompts and eval harness.
When is GraphRAG not worth it?#
When the corpus is small, entities do not recur across documents, queries are mostly local lookups, or docs change so often that extraction cost dominates.
Related Topics#
- RAG Patterns: the broader pattern family GraphRAG belongs to
- Embedding Pipeline at Scale: the index-time infrastructure GraphRAG reuses
- Vector Databases: the storage layer GraphRAG sits next to, not instead of
- Graph Databases: the storage layer GraphRAG sits on top of
Quick reference#
GraphRAG augments retrieval with an LLM-extracted knowledge graph. It targets multi-hop and global questions that vector RAG cannot answer. This page is the cheat-sheet view; see the deep dive for the pipeline and the Cypher example.
Mental model#
Vector RAG retrieves passages. GraphRAG retrieves paths plus passages. The graph is built offline by an LLM that reads each chunk and emits typed entities and relationships; queries traverse the graph or map-reduce over pre-computed community summaries.
Why it exists#
- Vector RAG fails on multi-hop questions because cosine similarity does not follow edges.
- Vector RAG fails on global questions because no chunk summarises the corpus.
- A graph makes both shapes tractable.
Pipeline stages#
- Chunk the docs.
- Extract entities + relationships per chunk with an LLM, using a fixed typed schema.
- Dedupe entities by normalised name + embedding similarity.
- Load into a graph database (Neo4j, Memgraph, or in-memory NetworkX).
- Detect communities with Leiden (or Louvain) at multiple resolutions.
- Summarise each community with an LLM; store the summary indexed by community ID.
- Route queries at serve time: local entity traversal vs global summary map-reduce vs vector fallback.
Query routing#
| Query shape | Path |
|---|---|
| Anchored on a specific entity | Entity linker, then bounded graph traversal (1 to 3 hops) |
| Corpus-wide synthesis | Top-K community summaries, then map-reduce |
| Lookup with no clear entity | Fallback to vector RAG |
| Thin context from primary path | Backstop with vector RAG |
Microsoft GraphRAG: load-bearing contribution#
The 2024 paper's main idea is pre-generated community summaries. Indexing pays the cost once; corpus-wide questions become cheap at query time. Hierarchical Leiden communities give the router a granularity knob.
Cypher traversal example#
MATCH (p:Person)-[m:MODIFIED]->(s:Service)<-[:OWNS]-(t:Team {name: 'Platform'})
WHERE s.domain = 'billing'
AND m.timestamp > datetime() - duration({days: 90})
RETURN p.name, s.name, m.evidence
ORDER BY m.timestamp DESC LIMIT 25
Three relationships in one query; evidence pointers hand chunks to the generator.
Trade-offs vs vector RAG#
| Dimension | Vector RAG | GraphRAG |
|---|---|---|
| Index cost | Low | High (LLM call per chunk) |
| Index time on 1M docs | Hours | Days |
| Query latency | 50 to 200 ms | 200 ms to 1 s |
| Multi-hop quality | Poor | Strong |
| Global synthesis | Very poor | Strong |
| Freshness story | Easy | Hard |
| Maintenance | Index only | Schema, prompts, dedup, summaries |
When to use what#
- Vector RAG: small corpus, lookup-style questions, fast iteration.
- Hybrid (vector + BM25 + rerank): jargon-heavy corpora; production default.
- GraphRAG layered on hybrid: entity-dense corpora with multi-hop and synthesis queries.
- GraphRAG only: rare; mostly small expert corpora.
When to skip GraphRAG#
- Cannot name 10 entity types up front that cover the corpus.
- Docs change minute to minute; extraction cost dominates.
- Queries are 95% local lookups against clean docs.
- Latency budget is sub-100 ms end to end.
Failure modes#
- Extraction noise: typos, hallucinated relationships. Mitigate with strict schemas and confidence thresholds.
- Stale graph: re-extract changed docs; re-summarise only affected communities.
- Schema drift: version the schema; shadow-index during extractor model upgrades.
- Summary staleness: stamp summaries with the graph version; recompute past a threshold.
- Router miscalibration: benchmark routing decisions per release.
Evaluation#
- Recall@K on local queries.
- End-to-end answer quality (LLM-as-judge or human).
- Synthesis quality on global queries (human eval).
- Entity precision / recall against a gold set.
- Community coherence (do summaries describe coherent topics).
Tooling#
- Microsoft GraphRAG: the open-source reference pipeline (Python, prompts, eval).
- LlamaIndex KnowledgeGraphIndex: lighter-weight alternative.
- Neo4j + APOC: storage and traversal.
- Memgraph: faster read path; good for large traversals.
- NetworkX: prototyping only.
Refs#
- Microsoft Research, "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" (2024)
- Edge, Trinh et al., GraphRAG open-source release notes
- Leiden community detection: Traag, Waltman, van Eck (2019)
- Neo4j Cypher reference
Interview prompts#
- Explain why vector RAG fails on multi-hop questions.
- Walk the GraphRAG indexing pipeline end to end.
- Describe how Microsoft GraphRAG answers a global question without scanning the corpus.
- Trade-offs: when would you not use GraphRAG?
- How do you keep the graph fresh as docs change?