The AI Analytics Bill Your CFO Will Eventually See
Your team just asked the AI to analyze publication trends across your research database. The system retrieved 50 articles, passed them to the model, and produced a fluent, confident summary. But your database has 16 million records. The system did not tell you it only looked at 50 of them. The answer came back anyway. Authoritative. Incomplete. Structurally unverifiable.
This is how most enterprise AI analytics actually works right now. Not because the engineers made a careless choice, but because the dominant architectural pattern, retrieval-augmented generation, was designed for lookup queries and then stretched to cover analytical work it was never built to handle. When someone asks "show me the trend in AI research funding over the last decade across our entire dataset," RAG treats that like a search query. It retrieves a slice of records, feeds them to the model, and produces an answer that reflects maybe 0.0003% of the underlying data. Nobody flags the coverage gap. The report goes out.
Ivan Dobrovolskyi, a researcher at Walmart Global Tech, published a paper this year addressing both the cost problem and the hallucination problem in a single architectural move. The system is called RES: Reasoner-Executor-Synthesizer. Across 100 benchmark runs on a scholarly database ranging from 42,000 to 16.3 million articles, RES maintained a mean token cost of 1,574 tokens regardless of dataset size. The naive RAG baseline cost 5,934 tokens on just 50 articles. At 16.3 million articles, the naive approach would extrapolate to approximately 1.9 billion tokens. The architecture does not just reduce cost: it makes the cost constant, so that analyzing 16 million records costs the same as analyzing 50.
The paper also formally proves, not claims, that this O(1) token complexity holds as a mathematical property of the design. That proof matters. It means the cost guarantee is not a benchmark artifact that degrades under different conditions. It is structural.
Why Every Prior Approach Still Gets the Bill Wrong
The problem with RAG at scale is not the retrieval step. It is what happens after. Every retrieved document enters the context window. More records, more tokens, higher cost, higher hallucination risk. The research on this is consistent: LLMs are more likely to generate unsupported claims when processing longer contexts, particularly when relevant information is buried among irrelevant passages. Adding more records to fix coverage makes the hallucination problem worse at the same time it makes the cost problem worse.
The approaches designed to fix this run into the same wall. Iterative retrieval and re-ranking reduce the volume of text sent to the model, but they do not eliminate the linear scaling relationship. Chain-of-thought prompting, ReAct agents, Toolformer: each tool call appends its result to a growing context. The context window keeps expanding with every step. Program-aided language models get closer by delegating computation to code, but the code results still return to the LLM for interpretation. Each approach reduces the problem. None of them breaks the linear relationship between data volume and token cost.
Text-to-SQL comes closest in spirit, but it is limited to structured databases with predefined schemas. RES generalizes the core idea and applies it to any data source with aggregation capability, with a formal guarantee attached.
The Strict Information Boundary That Makes O(1) Possible
RES is built around one principle that sounds simple and turns out to be difficult to enforce: the LLM never sees raw data records. Ever. Not at retrieval time, not at synthesis time, not anywhere in the pipeline.
This is enforced by separating the pipeline into three layers with zero overlap in responsibilities:
| Layer | What It Does | LLM Token Cost |
|---|---|---|
| Reasoner | Parses natural language query into a structured JSON execution plan | ~420 tokens, O(1) |
| Executor | Runs deterministic aggregation queries against the data source, outputs a compact statistical summary | 0 tokens |
| Synthesizer | Converts the fixed-size statistical summary into human-readable narrative and visualization | ~1,160 tokens, O(1) |
The Executor is the architectural keystone. It performs all data operations, counting, aggregating, filtering, sorting, using conventional code and API calls. No model is involved. It produces a compact JSON summary of roughly 500 to 800 tokens regardless of whether the underlying dataset has 42,000 records or 16.3 million. The Synthesizer then receives only that compact summary. It has no access to the underlying data. It cannot describe a data point it has never been shown.
The hallucination guarantee here is not a mitigation strategy. It is a construction property. The Synthesizer can only describe the verified statistics provided by the Executor, which the paper argues is a stronger guarantee than post-hoc fact-checking, retrieval-grounded generation, or self-consistency verification.
The Synthesizer may still exhibit linguistic imprecision, and the paper acknowledges this. The guarantee is specifically about data hallucination: fabricated counts, invented trends, confabulated comparisons. Those are eliminated by architecture, not by prompting.
What the Benchmark Actually Measured and What It Did Not
The empirical validation ran 100 queries across 20 distinct query types, spanning trend analysis, comparisons, rankings, and general statistics. The data source was the Crossref REST API, 130 million scholarly articles, queried live. Five runs failed due to API timeouts, not LLM failures. The 95 successful runs produced a mean of 1,574 tokens with a standard deviation of 259 tokens. The tight variance matters: this is a predictable cost, not an average that hides wide swings.
The comparison baseline deserves honest scrutiny. The naive RAG baseline sent 50 raw article records directly to the same LLM, Claude Sonnet, and produced a mean of 5,934 tokens. That 73.5% token saving is real. The 1.2 million times cheaper figure at 16.3 million articles is an extrapolation based on the O(n) scaling relationship, not a directly measured result. The paper is transparent about this, showing it as a log-log scaling projection rather than a measured data point. The extrapolation is mathematically sound given the formal proof of O(n) for naive RAG, but executives should treat the specific ratio as illustrative of order of magnitude rather than a precisely measured empirical outcome.
Also important: the paper does not compare RES against state-of-the-art RAG variants, iterative retrieval, compressed RAG, or re-ranking pipelines. The baseline is intentionally simple. Whether RES outperforms a well-engineered compressed RAG system on a 10,000-record dataset is not answered here. The paper's contribution is the O(1) proof and the hallucination-by-construction guarantee, both of which hold regardless of the baseline chosen.
How the Pipeline Actually Runs
The sequence is sequential and fully automated. Understanding each handoff makes the O(1) property intuitive rather than just asserted:
-
The Reasoner receives the user's natural language query plus a fixed system prompt. It identifies intent, whether trend analysis, comparison, ranking, or statistics, along with subjects and temporal constraints. It outputs a structured JSON query plan. Its token cost is bounded by query length, not dataset size.
-
The Executor takes that query plan and calls the data source using aggregation-only requests. For ScholarSearch, this means Crossref API calls with count-only and faceted query parameters, never downloading individual article records. The Executor processes all n records using conventional code, which is O(n) in CPU terms, but CPU cycles cost orders of magnitude less than LLM tokens. The output is a compact JSON summary, fixed size by design, containing data points, totals, and metadata.
-
The Synthesizer receives only that compact summary. It produces a human-readable narrative and a chart configuration for visualization. Its context window contains the fixed system prompt plus the fixed-size Executor output. Neither term grows with n. Token cost is O(1) by definition.
The ScholarSearch implementation exposes real-time callbacks so users can observe each layer's inputs and outputs as they occur. This transparency feature means the hallucination-free property is verifiable without inspecting source code. For compliance-sensitive applications, that observability is not a nice-to-have. It is what makes the guarantee meaningful.
Where This Architecture Changes the Economics
Consider a financial institution running trend analysis across a transaction database of 50 million records. Under a naive RAG approach, even retrieving a 10,000-record sample per query and passing it to an LLM produces token costs that accumulate quickly across hundreds of analyst queries per day. At scale, that is not a technology problem. It is a budget constraint that limits which questions analysts can afford to ask.
The same query under RES runs the aggregation deterministically, produces a 600-token summary, passes it to the Synthesizer, and returns an answer for roughly 1,574 tokens total. The analyst asks ten more questions. The cost is the same each time. The coverage is complete, not sampled. And the output cannot contain fabricated trend lines or invented figures, because the Synthesizer was never shown raw transaction records in the first place.
The paper explicitly names financial reporting over transaction databases, healthcare analytics over electronic health records, IoT sensor analysis, and log analysis for observability platforms as direct applications. These share a structural property: large datasets, analytical query types (trends, comparisons, rankings), and regulatory or operational environments where hallucinated outputs carry real consequences. The fit is direct in each case.
The hard boundary of the architecture also applies here: RES does not help when the query is "summarize the methodology section of this specific report" or "extract the key argument from document 47." Those queries require reading individual documents. The paper explicitly defers those cases to focused RAG or long-context models. Executives deploying RES need to classify query types before choosing architecture, not after.
Deploying the Pattern Without Waiting for a Perfect Data Source
The code is open-source under an MIT license, available now. The architecture itself is the deployable artifact. ScholarSearch is the existence proof, not the product.
The deployment question is almost entirely an Executor question. The Reasoner and Synthesizer are standard LLM API calls with fixed prompts. The Executor is the layer that must be engineered for each data source, and its requirements are specific: the data source must support aggregation queries. If the source has built-in faceted search or SQL-style GROUP BY operations, the Executor is a thin wrapper. If not, the Executor must process records programmatically to produce the aggregated summary, but the LLM token cost remains O(1) either way. The computational cost of the Executor shifts, not the guarantee.
Three questions determine whether a use case fits the architecture:
- Can the user's questions be answered from aggregated statistics, counts, trends, rankings, comparisons, rather than from reading specific documents?
- Does the data source support aggregation queries, either through built-in API capabilities or through programmatic processing?
- Is the query volume high enough that token cost predictability and hallucination prevention at the data layer justify the engineering investment in the Executor?
If all three are yes, the architecture is directly applicable. If the first is no, a different architecture is the right choice, and RES's authors say so explicitly.
The future work the paper identifies is telling: multi-source federation, a caching layer for repeated statistical queries, and formal automated verification of the hallucination-free property. None of these extend the core architecture. They industrialize it. The pattern is stable enough that the interesting engineering work is now at the edges, not the center.
The deeper implication is this: the AI cost problem at enterprise scale was never really a model problem. Models get cheaper every year. The problem was that the dominant architecture made cost a function of data volume, which meant the bill grew with every new record added to the database. RES breaks that relationship. Once cost is O(1), the question of what to analyze is no longer constrained by how much it costs to ask.
Written by the Agents Applied editorial team. Research extraction conducted against the published paper. Extrapolated figures are noted as such where they appear.