GraphRAG & Relationship Questions

Every technique so far in this course shares one underlying assumption: the answer to a question exists somewhere as a passage, sitting in some chunk, and the entire job of the system so far has been finding it efficiently. Two query classes break that assumption entirely. “How is the Meridian acquisition connected to our Berlin vendor contracts?” has its answer spread across a chain of documents, none of which individually mentions both ends of that chain. “What are the recurring risk themes across all vendor contracts?” has its answer in the aggregate across hundreds of documents, present in no single passage at all, no matter how well you search. Top-k retrieval, however carefully tuned by everything in Modules 3 and 4, samples fragments; these two question classes need connections and wholes instead, which is a fundamentally different retrieval problem.

GraphRAG is the 2024-25 answer that matured into a genuine, if situational, 2026 production option, and this lesson covers both how it works technically and, just as importantly for your actual decision-making, when it genuinely isn’t worth adopting.

The pipeline: build understanding at index time

Microsoft’s GraphRAG, open-sourced in 2024 and heavily iterated since, with lighter, cheaper descendants like LightRAG and HippoRAG trading some depth for meaningfully lower cost, moves the expensive comprehension work from query time, where users are waiting, to index time, where nobody is:

1. Entity and relation extraction. An LLM pass runs over every chunk, pulling out entities such as people, organizations, projects, and systems, along with typed relationships between them, for instance “Meridian Corp acquired by us” or “Berlin office supplied by Vendor GmbH.” This is the big cost center in the whole pipeline: an LLM call per chunk, roughly the same order of magnitude as the contextual retrieval enrichment from Lesson 8, but with meaningfully heavier prompts since extraction requires more careful, structured reasoning than a simple situating sentence does.
2. Graph construction with deduplication. “Meridian,” “Meridian Corp,” and “the Meridian acquisition” must all merge into a single graph node representing the same real-world entity, or the graph fragments into disconnected islands that defeat the entire purpose. Embedding-similarity matching plus LLM adjudication for ambiguous cases handles this merging step. Skimp here and the resulting graph is close to useless, since multi-hop traversal depends entirely on entities being correctly unified across mentions.
3. Community detection. A clustering algorithm, Leiden being the typical choice, partitions the constructed graph into densely connected neighborhoods, recursively, producing a genuine hierarchy: micro-communities capturing a single project and its immediate people, all the way up to macro-communities capturing an entire department’s activity.
4. Community summaries. An LLM writes a summary for every single community, at every level of that hierarchy. These summaries are themselves indexed separately. This is the precomputed aggregation step that makes global, whole-corpus questions like the risk-themes example above actually answerable at query time.

flowchart LR
    CH["Chunks\n(from Module 3)"] --> EX["LLM entity +\nrelation extraction"]
    EX --> G[("Entity graph\nnodes: people, orgs, systems\nedges: typed relations")]
    G --> CD["Leiden community\ndetection (hierarchical)"]
    CD --> CS["LLM community\nsummaries, per level"]
    CS --> IDX[("Summary index\n+ graph store")]

    style CH fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style EX fill:#fff7ed,stroke:#f59e0b,color:#0F172A
    style G fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style CD fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style CS fill:#fff7ed,stroke:#f59e0b,color:#0F172A
    style IDX fill:#f0fdf9,stroke:#0D9488,color:#0F172A

At query time, this precomputed structure serves two genuinely distinct query modes, and it’s worth being explicit that both require their own indexing investment, since one is not automatically usable in place of the other:

• Local mode, for multi-hop lookups: resolve the query’s mentioned entities to their graph nodes, expand outward a hop or two along the graph’s edges, collect the chunks attached to those nodes and edges as evidence, then generate an answer from that collected evidence. The graph traversal itself is the multi-hop reasoning here, executed in milliseconds via edge-walking instead of requiring several chained LLM calls to accomplish the same reasoning.
• Global mode, for sensemaking questions: map the query across the relevant community summaries at an appropriate level of the hierarchy, then reduce those partial per-community answers into one final synthesis. “Risk themes across 400 contracts” becomes a map-reduce operation over a few dozen precomputed summaries rather than an impossible attempt to read all 400 contracts fresh at query time.

The honest cost accounting

GraphRAG demos brilliantly in a conference talk, almost every time. The bill arrives afterward, in three installments that the demos conveniently skip over entirely.

Indexing cost. LLM extraction running over every single chunk in your corpus, entity resolution work to merge duplicate mentions, and summary generation over every community at every level of the hierarchy. On a genuinely large corpus this adds up to thousands of dollars and real, non-trivial pipeline engineering complexity, all of it paid before the very first query is ever answered.

Update cost. This is the sharp one, and the one that catches teams off guard after the initial build looks successful. Lesson 6 built incremental sync where an updated document swaps its chunks in cleanly within minutes. In a graph, that same updated document’s entities participate in communities whose summaries are now, by definition, stale the moment the document changes, and re-summarizing correctly cascades upward through the entire hierarchy those entities belong to. Incremental graph maintenance techniques have genuinely improved, this was a major focus of research and engineering effort through 2025, but graph maintenance remains fundamentally harder than the simple chunk upsert pattern from Module 2. A graph built over a fast-changing corpus becomes a permanent maintenance treadmill rather than a one-time investment.

Wrongness cost. Extraction errors compound structurally in a graph in a way they don’t in flat chunk retrieval. A mis-merged entity, “John Smith” from the legal team’s documents accidentally merged with a different “John Smith” from sales, silently fuses two genuinely unrelated subgraphs together, and every answer touching either person afterward inherits that confusion, presented with exactly the same confidence as answers built from correctly merged entities.

So the adoption razor worth applying before committing engineering time to this: measure what fraction of your real production traffic is actually relationship-shaped or global-shaped, using the query logs you’ve been collecting since Lesson 2’s logging recommendation. In most enterprise assistants that fraction turns out to be a genuine few percent, not the dominant traffic pattern the demos might suggest. The pragmatic ladder that follows from this measurement looks like: handle occasional multi-hop questions with the agentic chained retrieval covered in Module 6, at zero standing indexing cost since it only activates per query; handle “summarize this whole collection” requests with periodic, scheduled per-collection summary documents fed back into the normal chunk index, essentially a one-cron-job GraphRAG-lite that covers a surprisingly large share of global queries without any graph infrastructure at all; and only adopt real graph indexing, with its real maintenance burden, when a specific corpus is high-value, relatively stable rather than rapidly changing, and demonstrably full of genuine relationship queries in your measured traffic. Legal entity webs, pharmaceutical research corpora, and intelligence-style analytical work clear that bar routinely and are exactly where GraphRAG shines. The average company wiki, in most measured deployments, does not clear it.

If you do adopt GraphRAG after this analysis, scope it deliberately: build the graph over the one specific collection that genuinely needs it, not the entire corpus indiscriminately, and run it alongside the standard hybrid stack from Module 4, with the router pattern from Lesson 14 deciding which machinery, graph or standard retrieval, each incoming query actually deserves.

That closes out query intelligence for this course. The stack now finds the right evidence for nearly any shape of question a real user might ask. Module 6 turns to what the generation model actually does with that evidence once retrieval hands it over: generation you can genuinely audit, citations users can actually check against the source, and the agentic loops that take over automatically when a single pass of retrieval isn’t quite enough to answer confidently.