When a Knowledge Graph Earns Its Keep

"What happens if service A fails while process B is running?" In a technical knowledge base, answering that takes the interaction between A and B, the failure handling in B, and the recovery mechanism in A. Those facts sit in the service A docs, the process B docs, and an integration spec.

Each document is relevant to one hop. None is highly similar to the full query. A flat index returns the overview pages for A and B, which sit close to the query as a whole, and misses the failure-mode pages that hold the answer.

GraphRAG grounds retrieval-augmented generation in a knowledge graph rather than a flat vector index. The same people get enthusiastic and then skeptical, often at different points in the same deployment, and both reactions are justified. The enthusiasm is a retrieval problem the graph handles well. The skepticism is the cost of building and keeping the graph.

Calling it a general quality upgrade to RAG gets it wrong. What decides it is whether multi-hop failure is your measured bottleneck, and whether the corpus is stable enough that the edges you extract still exist next month.

Where the flat index breaks

It embeds the query, embeds the chunks, and returns the top-k nearest neighbors. Fine when the answer lives in one chunk close to the query, and most factual lookups are that case. The multi-hop question is the exception: the answer needs two or more source documents, neither contains it alone, and the two are not necessarily in the top-k neighbors of the query.

What the graph encodes

Entities become nodes. Relationships become typed, directed edges. Nodes can be services, processes, API endpoints, data schemas, or other domain concepts, and edges encode how they relate: "service A calls service B," "process B reads from schema C," "endpoint D fails with error E when condition F."

A query arrives and the system identifies the entities, locates them in the graph, and retrieves by traversal rather than similarity. For the service A / process B failure, it finds A and B, walks the edges connecting them, and pulls the documents on that path. The documentation comes from the structure of the domain, not from the statistical proximity of text.

A three-hop question — "what are the downstream effects of changing schema C, given that process B reads from C and service A calls process B" — is answered by walking A to B to C and retrieving the documents on each node and edge. The answer is in the path. No single document has to resemble the full query.

Building it

The traversal is the part that looks clean in a diagram. Construction decides whether the system works, since the graph can only retrieve through relationships it captured. You extract entities, resolve co-references, and infer relationships. Each step is error-prone and the errors compound.

On technical documentation, entity extraction is typically in the 85–90% range for a well-tuned system. So a 10–15% miss rate leaves a significant fraction of entities absent from the graph or imprecisely represented. Co-reference is a separate failure: the "firm" in document A and the "company" in document B have to land on the same node if they refer to the same organization, which is one of the harder problems in NLP, particularly across documents that use different terminology for the same concept.

Errors produce fragmented entities, multiple nodes for one thing, none holding the complete information.

Typing the edge is harder. "Service A monitors process B" and "service A depends on process B" are different relationship types with different implications, and on specialized technical documentation relationship extraction accuracy can fall below 70%. The 85–90% and the 70% are typical bands. No labeled slice or named extraction run stands behind either. Traversal quality is bounded by the noise in the map that results.

Schema is a second, quieter construction error. Generic relationship types such as "related_to," "mentions," or "depends_on" make traversal a more expensive neighborhood search. Types that are too fine become brittle, because extraction quality drops as the model distinguishes labels that human annotators would debate. The useful middle is domain-specific and operational, with enough structure to answer the questions you ask.

Keeping it current

A graph built from a static corpus is constructed once and queried. A living corpus, where documentation changes with releases, policies update, and integration specs evolve, means maintenance: entities change, relationships change, documents are deprecated.

A flat index does not cascade this way. You re-embed the changed documents, each embedding is independent, and the rest of the index stays put. A graph has structural dependencies. If the specification for service A changes its relationship to process B, that edge has to be updated and every downstream relationship that depends on it re-evaluated. Teams that try it at scale often find the graph degrades faster than they can maintain it, particularly on fast-moving engineering documentation, which is why GraphRAG ships less often than a clean multi-hop run would suggest.

When it pays

The query distribution needs a significant fraction of multi-hop questions, and flat-index retrieval has to be measurably failing on them. Only that measurement shows the retrieval architecture is the bottleneck rather than the model, the prompt, the chunking strategy, or a missing reranker. A hybrid index plus a reranker can close some multi-hop cases without the graph tax. If that stack already recovers the failures you measured, skip the graph.

The corpus has to be stable enough that maintenance is tractable. High-churn documentation is a poor fit, since relationships decay as fast as they are extracted. Reference documentation with defined versioning cycles gives the graph time to stay useful between updates. And the domain needs entity relationships that can be extracted reliably: news articles, customer support tickets and unstructured notes are usually a poor substrate, while technical documentation with named components, defined APIs, and explicit integration relationships is a better one.

If you cannot measure multi-hop failure on the current index, you do not have a graph problem yet. If you can, the test is small. Run extraction on a representative slice of the corpus, inspect the entities and relationships by hand, then replay the failed queries against the graph. Below about 80% extraction quality the noise eats the gain (80% is a planning floor of the same kind as the other bands, not a result from a named run), because missing and incorrect structure cause retrieval failures that offset the traversal wins. If the graph answers the failed queries cleanly, the architecture has earned its cost.

If the failures are diffuse (documents not indexed, stale sources, a weak model, a chunk-size mismatch), the graph does not touch them, and you pay its cost anyway. GraphRAG is a specialized architecture for one failure mode, multi-hop questions that need cross-document structure. The price is construction complexity, operational overhead, and sensitivity to extraction quality. If your failures concentrate in the multi-hop case and the corpus is stable and structured enough to support a maintained graph, the investment can make sense.