The Enterprise AI Problem Isn't the Model. It's Everything Around the Model.
The job-matching algorithm your HR platform has spent three years tuning lives in a container on a GPU cluster. The resume parser is a separate fine-tuned model. The relational database holding job postings requires a specific SQL dialect. The graph database mapping skill taxonomies speaks a different query language entirely. And the LLM your team just integrated to handle conversational career advice has no idea any of those things exist.
So the LLM does what LLMs do when they lack context: it improvises. It answers questions about "data scientist roles in the SF Bay Area" without querying the actual jobs table, because nobody built the connection. Or it queries only one source when the correct answer requires three. Or it produces an output that the next agent in the chain cannot consume because nobody defined a transformation layer. The workflow completes. The answer is wrong. Nobody knows why.
Researchers at Megagon Labs published a blueprint architecture paper addressing this class of failure directly. The paper, from Eser Kandogan and five colleagues, does not propose a new LLM or a new fine-tuning technique. It proposes a systems architecture for compound AI: a structured framework in which LLMs are one type of agent among many, coordinated through observable data streams, discoverable through registries, and constrained by explicit cost, accuracy, and latency budgets. The paper's central argument is that the enterprise AI coordination problem cannot be solved by a better model inside a broken architecture. That reframe has significant operational implications for any organisation currently building on top of LangChain, AutoGen, or LlamaIndex.
Why Letting the LLM Orchestrate Everything Is the Wrong Default
The standard deployment pattern for enterprise multi-agent AI today is: give the LLM a list of tools, let it plan which tools to call, and pass the outputs between them as text. This works cleanly in demos. It degrades at enterprise scale for four specific reasons the Megagon Labs paper identifies with precision.
First, existing frameworks have no systematic discovery mechanism. When your enterprise has hundreds of APIs, dozens of databases, and a library of fine-tuned models, the LLM cannot know what exists unless someone hardcodes that knowledge. Hardcoded tool lists go stale immediately and do not scale.
Second, the data flow between agents is almost entirely textual. An agent that produces a structured SQL result passes it as a text blob to the next agent, which has to re-parse it. An agent that needs a graph database query result receives it as a string representation, losing schema information in transit. The architecture assumes a uniform data type that enterprise data infrastructure does not have.
Third, there is no multi-objective optimization layer. LLM-focused frameworks optimize for accuracy or cost within a single model call. Nobody is estimating and tracking the total cost, latency, and quality budget across a five-agent workflow that spans a cloud LLM, a local fine-tuned model, and two database queries.
Fourth, and this is the one that matters most for regulated industries: there is no first-class observability. Data flows between agents implicitly. If a decision was made incorrectly, you cannot replay the data flow that produced it, because that flow was never recorded as a structured artifact.
Streams as the Primitive That Changes the Observability Equation
The Megagon Labs architecture is built around one structural choice that differs from everything else in the current framework ecosystem: it treats coordination as a data problem, not a prompting problem.
The central orchestration concept is streams. A stream is a sequence of messages, containing either data or instructions, that can be produced, distributed, monitored, and consumed by any component in the system. Every agent reads from input streams and writes to output streams. Every instruction from a task coordinator to an agent travels through a stream. Every data transformation between agents is mediated by streams.
The key architectural decision is that streams are modeled as data structures and treated as first-class data resources within the system. This is not a messaging queue in the background. The entire interaction history, every data message and every control instruction, is a structured, queryable artifact. You can audit what data triggered which agent. You can replay a workflow to understand a failure. You can observe in real time where a plan is executing versus where it is stalled.
This has consequences that extend well beyond debugging:
- Compliance and audit trails become structural rather than retrofitted. Every agent action is traceable to a stream event, which is traceable to a user request, without additional logging instrumentation.
- Decentralized agent activation becomes safe. Agents can monitor stream tags and activate autonomously when relevant data appears, because their activation condition is explicit and recorded, not emergent from an LLM prompt.
- Human-in-the-loop checkpoints have a natural insertion point. A task coordinator can pause execution, write a confirmation request to a stream, and resume only when the user's response arrives. The pause state is durable.
Registries Solve the Discovery Problem That Hardcoded Tool Lists Cannot
The second structural innovation in the Megagon Labs blueprint is the pairing of an agent registry and a data registry. These are not configuration files. They are searchable, vector-indexed metadata stores that the task planner queries at runtime to construct execution plans.
The agent registry stores descriptions, input/output parameter schemas, stream subscription rules, deployment configurations, and historical usage logs for every available computational entity in the system. That includes LLMs, fine-tuned models, search interfaces, and legacy APIs. Any enterprise asset that can process input and produce output can be registered as an agent. The registry supports both keyword search and vector-based search over learned representations of agent metadata and usage history, which means the task planner can find relevant agents for novel task types it has not explicitly been configured for.
The data registry operates at equivalent depth for data sources. It stores metadata at every level from the lakehouse down to individual tables, including schema details, data descriptions, available indices, query history embeddings, and connection specifics. It covers relational databases, document collections, graph databases, and key-value stores under a single discovery interface.
Without both registries, the task planner is navigating enterprise infrastructure blind. With them, it has a searchable, continuously updatable map of every asset available to it, with enough metadata to make informed decisions about which assets to use and in what combination.
The Planning Problem the Paper Doesn't Pretend to Have Solved
The Megagon Labs paper is careful about what it claims. The open research questions section is unusually candid, spanning five categories: agents, data and operators, planning, optimization, and reliability. This honesty is worth examining, because it tells executives what the blueprint is and is not.
On planning specifically: the paper explicitly cites literature showing that LLMs alone cannot reliably solve planning problems. The task planner component is architected to decompose user requests into agent execution graphs, but the paper acknowledges it does not yet have a complete answer for how to verify and constrain LLM-generated plans, how to perform planning across multi-modal data sources, or how to incorporate execution feedback into plan refinement.
On reliability: chaining multiple nondeterministic agents compounds failure risk. The task coordinator monitors execution budgets and can abort and replan, but the fault-tolerance mechanisms for the agents themselves remain an open design question.
On data governance: the paper explicitly raises the question of how to enforce data access privileges across agents with different permission levels and then notes it is unresolved. For any organisation in a regulated industry, this is not a minor open question. It is a deployment blocker until answered.
This is a blueprint with working proof-of-concept implementation, not a production framework ready for procurement. Treat it accordingly.
How the Architecture Actually Executes a Workflow
The paper implements the blueprint as "Agentic Employer," a conversational interface for HR hiring managers at a fictional mid-size firm called YourJourney, which has roughly one million job seekers and employers in its platform, along with fine-tuned models for skill extraction and matching, relational databases for job postings, and document collections for resumes. The workflow for a query like "show me the top data scientist candidates for our Mountain View role" executes in the following sequence:
- The user input is emitted into a stream and a session is established, scoping the collaboration context for all participating agents.
- The task planner, itself running as an agent subscribed to the user input stream, interprets the request, queries the agent registry to identify suitable agents, and produces a directed acyclic graph plan: Intent Classifier, then NL2Q agent, then SQL query executor, then summarizer.
- The task coordinator picks up the plan from the stream, estimates costs via the optimizer, and initiates each agent in sequence by writing control messages to their input streams.
- The data planner is invoked, either by individual agents or by the coordinator, to decompose the data retrieval: identify SF Bay Area cities from an LLM call, map relevant job titles from a graph database containing the skill taxonomy, then apply a select operator on the relational jobs table.
- Each agent writes its output to an output stream. The coordinator monitors execution against the budget, tracking time, cost, and quality. If thresholds are exceeded, it can abort and request user confirmation before replanning.
- The final summarizer agent renders results to the user interface, with complex JSON outputs handled by interactive renderers that can generate UI components declaratively.
The entire flow, from every data message to every control instruction, is recorded in streams. A compliance reviewer can query the audit trail. A developer can replay a failed workflow from the stream history.
What This Architecture Implies for Decisions Being Made Right Now
Most enterprises evaluating compound AI today are making a platform choice between framework-level abstractions: LangChain, LlamaIndex, AutoGen, or custom orchestration built on top of a foundation model provider's API. The Megagon Labs blueprint does not fit cleanly into that choice set, because it is arguing for a different level of abstraction entirely, one borrowed from distributed systems and database architecture rather than from AI model tooling.
The paper's most significant strategic implication is this: if you are building compound AI on frameworks that route everything through text and give the LLM implicit control over coordination, you are accumulating architectural debt that will become expensive to unwind as your agent count, data source count, and compliance requirements grow. The streams-and-registries approach is harder to stand up initially. It requires distributed systems engineering, metadata curation for every data source and API, and container orchestration that your AI team may not currently own. But it produces a system that is auditable, extensible, and optimizable in ways that text-flow architectures are not.
The cost of being on the wrong architecture in 18 months is not a performance gap you can close by swapping in a better model. It is a structural rebuild at the point when your compound AI system has become load-bearing for business decisions.
The blueprint is not finished. Several of its most important components, particularly the planner, optimizer, and data governance layer, remain open research problems. But the architectural direction it points in is the one that enterprise-scale deployment actually requires. The question for any organisation building compound AI today is not whether this architecture is production-ready. It is whether their current architecture would survive being held to its standard.
Agents Applied is a weekly newsletter for executives and strategists navigating the operational reality of AI deployment.