The GraphRAG Systems Making Your Enterprise AI Worse Than Simple Search
A team in your organisation deploys a GraphRAG system on clinical protocols. The sales pitch was compelling: graph-based retrieval understands relationships between concepts, follows multi-hop reasoning chains, returns contextually richer answers than dumb vector search. The system goes live. Retrieval recall climbs. Everyone is satisfied.
What nobody checks is the relevance score. Graph retrieval is returning more content, yes. But it has quietly dropped from 62.9% relevance to 38.5%. The context feeding your LLM is now noisier than what vanilla RAG would have produced. The model generates fluent, confident answers grounded in a mess of contradictory and thematically irrelevant facts extracted in isolation from documents that were never connected to each other during the build phase. The graph that was supposed to be the advantage became the liability.
This is not a hypothetical. Researchers from Xiamen University and Jilin University documented this exact dynamic in a paper accepted to KDD 2026 — one of the premier venues for data mining research. Their preliminary study on a medical benchmark shows GFM-RAG achieving 84.3% recall versus 71.8% for standard RAG, while its context relevance collapses to 38.5% against 62.9%. More retrieval, worse context: the graph made things worse, not better. Their response is MemGraphRAG, a framework that addresses the root cause of this failure rather than patching the retrieval layer. Across five benchmark datasets and four evaluation tasks, MemGraphRAG achieves a 59.25% overall average accuracy against the previous best of 57.15% from LinearRAG, and retrieves in 0.061 seconds per query, the fastest of any system tested.
Why Every Existing GraphRAG System Is Building the Wrong Graph
The failure is not subtle once you see it. Every major GraphRAG pipeline, including Microsoft's GraphRAG, HippoRAG, LightRAG, and GFM-RAG, extracts knowledge from document chunks in isolation. Each chunk is processed independently. The extracted triples go directly into the graph without any awareness of what other chunks have already contributed.
This architecture produces three compounding defects. First, thematic irrelevance: triples extracted from isolated chunks often capture facts that are locally coherent but globally meaningless given the corpus's actual subject matter. Filtering out just 40% of low-frequency triples from a medical graph improved accuracy from 64.85% to 65.28%, a modest gain that reveals how much noise contaminated the original structure. Second, logical inconsistency: contradictory facts emerge within the same subgraph because no process compares what chunk 47 extracted against what chunk 12 already established. Third, structural fragmentation: entities appear as isolated nodes with no connections to related concepts across documents, which breaks the multi-hop reasoning chains that are the entire point of using a graph.
The paper's diagnosis is precise: these problems persist because existing pipelines derive knowledge from isolated local segments and lack a global perspective on previously processed context. The paradox this creates is stated plainly by the authors: GraphRAG systems frequently underperform naive RAG systems in many real-world applications, precisely because of the low quality of automatically constructed knowledge graphs.
Shared Memory Changes What Agents Can Know While Building
MemGraphRAG's central architectural move is introducing a Three-Layer Global Memory that persists across the entire document processing pipeline. Rather than discarding context after each chunk, every agent in the system reads from and writes to this shared memory throughout graph construction. The memory is not a log. It is a live knowledge repository with three interconnected layers:
- The Ontology Layer stores extraction schemas with their cumulative frequencies. A schema is a template-level pattern, such as (Person, born_in, Country), not a concrete fact. When a schema appears frequently across many chunks, it signals genuine thematic relevance. When it appears once, it is noise until proven otherwise.
- The Fact Layer stores the instantiated triples derived from confirmed schemas. Facts only graduate to active status when the schema governing them has been seen frequently enough to pass a stability threshold. This means no newly extracted triple can pollute the graph before earning it.
- The Passage Layer preserves the original source text segments linked to every fact. When two agents disagree about a contradiction, the evidence is retrievable. Resolution is not heuristic. It is adjudicated against the source.
Three specialised agents operate against this shared memory. The Extraction Agent populates all three layers simultaneously for each chunk. The Conflict Detection Agent runs a hybrid scan, combining vector similarity and symbolic matching, whenever a new triple is promoted to active status. The Conflict Resolution Agent retrieves the original provenance passages from the Passage Layer and adjudicates by comparing textual evidence directly, functioning, in the authors' framing, as a judge reviewing case files rather than guessing.
The probationary sandbox is the insight that makes this work: instead of trusting LLM extraction outputs immediately, the system treats them as unverified hypotheses. Schemas accumulate evidence across the corpus before any triples they govern are admitted to the graph. The graph evolves through an iterative extract-verify-modify cycle rather than a single pass.
The Construction Cost Nobody Is Talking About
The 0.061-second retrieval time is real and the paper demonstrates it cleanly. MemGraphRAG is roughly 26 times faster than HippoRAG, 35 times faster than HippoRAG2, and 181 times faster than LightRAG at query time. For any system handling thousands of daily queries against a stable document corpus, that difference in retrieval latency is operationally significant.
What the paper does not quantify is the offline graph construction cost. Three agents run on every document chunk. Conflict detection scans all previously activated triples every time a new triple achieves active status. Conflict resolution retrieves provenance passages and calls an LLM to adjudicate. This is substantially more LLM compute per document than single-pass extraction, and the paper provides no direct measurement of construction time or API cost relative to the baselines.
The architecture is optimised for corpora that are built once and queried many times. For a 10,000-document medical protocol corpus that changes quarterly, the construction cost is a one-time investment. For a corpus that is updated daily with new regulatory filings, the conflict scanning across an ever-growing fact set requires careful capacity planning that the published results do not address.
What the Graph Actually Looks Like When Memory-Guided
Graph quality metrics are rarely reported in RAG benchmarks because they are harder to interpret than accuracy scores. The paper reports them, and the structural differences are striking. On the G-Novel literary corpus, MemGraphRAG achieves an average clustering coefficient of 0.865 versus HippoRAG2's 0.657. On G-Medical, the average degree reaches 14.37 versus HippoRAG2's 13.31. MS-GraphRAG, which many enterprises are currently running, achieves an average degree of just 1.82 on G-Medical — a nearly tree-like structure with almost no cross-entity connectivity.
A clustering coefficient of 0.865 means the graph is densely triangulated: entities are not just connected to each other but connected through shared neighbours, which is exactly the structural property that enables genuine multi-hop reasoning rather than sequential hop traversal. This is not a cosmetic improvement. A graph that looks fragmented produces fragmented reasoning, regardless of how sophisticated the retrieval algorithm applied to it is.
Two bridging mechanisms create this density. Type-Based Bridging connects disjoint entities that map to the same high-level schema type, using the Ontology Layer to identify the shared category. Similarity-Based Bridging adds implicit edges between entity pairs whose vector representations exceed a configurable similarity threshold. Both mechanisms run against the global memory, not against individual chunks, which is why they can identify connections that isolated extraction would never surface.
Where It Gets Real: Clinical Protocol Retrieval
The G-Medical benchmark is built from NCCN clinical guidelines: dense, rule-based treatment protocols where the relevant facts are highly interdependent. A question about treatment eligibility for EGFR-mutant NSCLC requires connecting drug names, mutation classifications, treatment lines, and contraindications scattered across multiple protocol sections.
On this corpus, MemGraphRAG achieves 68.40% LLM-based accuracy, against HippoRAG2's 64.85% and GFM-RAG's 58.19%. The retrieval picture is equally clear: MemGraphRAG scores 90.42% complex reasoning recall against RAPTOR's 89.70% and HippoRAG's 83.80%, while maintaining 82.64% relevance against GFM-RAG's 33.06%. GFM-RAG's recall of 85.03% on this task costs it nearly 50 relevance points. A clinical decision support system built on GFM-RAG is retrieving a lot of protocol text and returning a lot of noise.
A clinical informatics team using MemGraphRAG instead would see two operational effects. The context fed to the generating LLM is both more complete and more relevant, which means fewer hallucinated treatment combinations and fewer gaps where the model fills missing evidence with confident inference. The 0.061-second retrieval latency also means this could operate as a real-time lookup within a clinical workflow rather than a batch process run overnight.
The caveat is that NCCN guidelines are text documents. MemGraphRAG currently handles only unimodal textual input. Clinical data that exists as structured tables, dosing charts, or embedded diagrams requires transcription to text before processing, and the authors acknowledge that transcription loses information. A full clinical deployment would need a preprocessing pipeline to convert non-text elements before the memory-based agents begin their work.
What Executives Running GraphRAG Systems Should Do Now
The first move is a relevance audit, not an accuracy audit. Most teams benchmark their RAG system on whether the final answer is correct. They do not separately measure whether the retrieved context was relevant to the question. The GFM-RAG example, 84.3% recall at 38.5% relevance, shows that a system can look healthy on recall while feeding the LLM context that is mostly noise. Pull the retrieval logs. Score relevance independently from accuracy. If relevance is lagging recall by more than 15 to 20 percentage points, the graph construction is producing noise at scale.
The second move is to test whether your current GraphRAG system is actually outperforming vanilla vector search on your own corpus. The KDD 2026 paper documents multiple cases where it does not. Run the comparison on a representative 500-question sample before assuming the graph is helping.
The third move is to evaluate MemGraphRAG against your actual document update frequency. The code is publicly available at github.com/XMUDeepLIT/MemGraphRAG and the construction uses GPT-4o-mini as the default LLM, so the experiment cost is manageable. But if your corpus updates continuously, prototype the construction pipeline on a static snapshot first and measure the LLM call volume before committing to a production rollout. The retrieval is fast. The build requires honest cost accounting that the paper does not supply.
The broader implication sits in what this research actually confirms: the theoretical advantages of knowledge graphs for enterprise retrieval are real, but they only materialise when the graph is built correctly. Every system that constructs graphs through isolated chunk extraction is, to varying degrees, undermining its own premise. The question is not whether to use graph-based retrieval. The question is whether your current graph is built well enough to earn it.
Agents Applied is a weekly briefing on applied AI research for executives and senior technologists. Each edition translates one significant paper into operational intelligence.