The Context Layer is a Useful Fiction: Why AI Governance Must Cover the Whole System

The enterprise AI stack has a favorite diagram: Data sits at the bottom, agents sit at the top, and a tidy "context layer" in the middle holds everything an agent needs to know. The picture is appealing because it implies a clean division of labor and an obvious home for governance. It also breaks down when treated as a standalone layer, because even highly governed data can produce wrong answers once an AI model is in the loop.¹

In a recent episode of AI Radicals,⁹ host Satyen Sangani sat down with Sanjeev Mohan, principal at SanjMo, former Gartner analyst, and author of Designing the AI-Driven Data Foundations,¹ to map the full stack from metadata to knowledge graphs. The conversation turned into a productive disagreement about where AI governance belongs. Their answers differ in emphasis and converge on one conclusion: govern the system, not a layer.

Why the context layer idea breaks down

Mohan builds the vocabulary from the bottom up:

• Metadata is data about data.

• Semantics translates that metadata into business language, including glossary terms, rules, and KPI definitions such as net retention rate.

• Context is the next step up, and it is where the engineering gets hard.

He describes it this way: "Now we are talking about giving AI this complete 360-degree view of a customer, a product, or a task. That is context." In this way, context blends structured and unstructured data, often in real time, and it can change from person to person for the same question.

Sangani's objection is to the architecture people draw around it. He calls the context layer a straw man: "There's this idea out there of this context layer, [which is] designed on this premise of microservices thinking that assumes a discreteness that just simply does not exist."

The problem is practical. A layer promises a stable, reusable body of knowledge that sits apart from the applications consuming it. Sangani argues that real AI systems do not divide that cleanly: "These are not independent layers. This is a system and an app that you have to think of as an encapsulated whole." He also names the temptation behind the diagram: "Building a layer is a sort of technologist dream, that I can capture all this information and it's sacrosanct. But it's all defined in the function of whatever the application happens to be, and the application is gonna drift constantly."

If the application drifts, whatever you captured in the layer drifts with it. Teams that treat context as a finished artifact will find it stale, while teams that govern it as part of a living system can keep it right. The business metadata at the base of that system, such as definitions, owners, and tags, typically lives in a data catalog. The same foundation also feeds the semantic layer, which is comparable with ontologies and the enterprise context layer (for more on on this, read the guide to semantic layer vs. ontology vs. enterprise context layer.)²

Where the stop sign belongs

Sangani grounds the debate in an example from a chief data officer at a major airline. Marketing teams building campaigns should not be able to use certain customer data, such as preference information, even though the company holds plenty of that information.

Now add an agent. If the agent has been instructed to build a fabulous marketing campaign, it will go looking for as much data as it can find. Eventually it hits the preference data.

Whose job is it to stop it? Sangani's answer is the data's: "The stop sign has to happen with the data itself, at the intersection where the things are happening." This is because no agent can carry every organizational rule in its head. "We don't necessarily have to know all of the rules. We just have to be told them," he says. This is the same logic that separates a dashboard from a product: "The person building the dashboard might just build a dashboard. The person building the product has to think about what are acceptable uses of this overlying thing."

The solution? Putting acceptable-use rules into governed data products and data governance policy, where they travel with the data to every consumer. It is also why Sangani frames data products as a form of context. The rules sit next to the thing they govern, so each new agent inherits them without being retrained or re-prompted. For more on why agents depend on that trusted foundation, see Alation's perspective on AI agents and trusted data.

Why clean data no longer guarantees a correct answer

Mohan agrees with the destination and adds a complication. For years, data governance rested on a simple bargain: "If my input was properly governed and clean, trusted, then my output would be exactly the same every single time, because it's deterministic." Models broke that bargain, because "even with the same data, [they] can produce different outputs at different times of the day." Governing inputs is no longer sufficient. Outputs need governing too, including harmful content and personally identifiable information.

Agents stretch it further. They reason, choose actions, and message one another, which generates new data that never existed in an operational system. Mohan calls it trajectory data and memory state, and it raises an audit question most governance programs have not faced: "If somebody says, 'I want lineage on how my multi-agent system rejected a loan,' I need to know what these agents were discussing on their message boards and what messages." His summary is compact: "That is, to me, agentic governance. It's input, output, and runtime." Alation's guide to governing agentic AI describes a similar shift, treating the agent as a governed runtime whose reads and writes need lineage and whose policies are enforced at decision time.³ Tracing any of that end to end depends on data lineage.

Indeed, the cost of weak controls is showing up in analyst forecasts. Gartner predicts that more than 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, or inadequate risk controls.⁴

Mohan offers a scenario that undercuts the case for any single-layer view. A team swaps one Gemini version for a cheaper Flash model and the rest of the system looks identical: "The rules are the same. The data is properly governed," yet the new model starts expecting a different prompt, and the application can break. "So that's why the governance has to be layered across data and AI to do its job," he concludes.

Research on a single model family shows how much a version change can matter. A Stanford and UC Berkeley study found that GPT-4's accuracy on one prime-number task fell from 84% to 51% between its March and June 2023 versions.⁵

The two positions are closer than they first appear. Sangani lists the many places a bad output can originate: the prompt, an outdated tool, the model, or an undocumented acceptable-use policy. His prescription is to write the correction back to wherever the fault lives, whether that is the data, the product, the semantics, or the agent. Mohan's scenario shows how a change in just one of those places can break a system that is otherwise fully governed. Both argue for governance-as-system, and Sangani says it directly: "You have to have the full chain governed to be able to guarantee that the agents are correct."

Data products: Data finally gets a version number

If governance has to span the whole chain, it needs a concrete home, and the data product is that home. By packaging data with its owner, rules, contracts, and lineage, a data product operationalizes AI governance: policies travel with the data to every agent and app that uses it.

Sangani describes it this way: a data product is a portable semantic layer that bundles business rules, data contracts, dimensions, metric definitions, and evals. So, with each data product you get: an owner, a versioned contract, quality enforcement, and lineage.⁶ Mohan adds the human-side framing: "One definition is that data product is a philosophy." In his telling, it means building any artifact with product management discipline: "There's a data product owner. It has a version number, and it's backward compatible. It's packaged with SLAs and contracts."

Both framings point at the same missing ingredient: is accountability over time. Mohan recalls years of building data warehouses where the team disbanded after launch, and nobody owned the errors that surfaced later. "A product has a life, like a beginning and end of a life, and so it's reusable. There's an owner." Sangani makes the reusability point from the other direction. Dashboards and quick AI-generated artifacts often fail the product test because their builders are thinking, "Let me solve my problem."

Today, the data product manager bridges business needs and data capabilities, pairing with a data steward much as a traditional product manager pairs with an engineering manager, and is responsible for making data products reusable; this reduces redundant workflows and conflicting versions of the same dataset.⁷ A data product operating model aligns the people, processes, and technology around that role. Sangani's field observation backs this up: the business knowledge and the technical requirements rarely sit in one head, "and so it tends to be this partnership."

Mohan likens data products to products: "Even now, when was the last time you heard data has a version number? It doesn't. A product does. My iPhone has a version number, not my data. But what you're doing with data products gives it that version number." A version number gives a correction somewhere to land. When an agent gets an answer wrong, you can ask which version of which rule it followed, and fix that version.

The moat is moving to semantics and business logic

The stakes extend beyond governance. Mohan sees the competitive ground in enterprise AI shifting under everyone's feet. Storage has been commoditized by open standards: "Iceberg for analytical, Parquet files, all that, S3-compatible storage." The interface is following. He points to Salesforce, whose Claudeforce partnership with Anthropic now lets sellers work with Salesforce data, workflows, and permissions from inside Claude, with the same answers available in Lightning and Slack.⁸ In Mohan's reading, "the UI goes to the AI tool."

What remains valuable is what the AI tool cannot supply on its own. "This is super exciting for me, to see how the consumption layer is getting democratized, but the moat is now shifting to the semantics," Mohan says, and he calls the middleware that links business logic and semantics "the most critical piece in this stack."

This is the strategic argument for treating governed data products as core infrastructure. When any model can sit on top and any storage engine can sit underneath, the organization's own definitions, rules, and contracts carry its competitive advantage. A data products marketplace is where those assets get discovered, trusted, and reused.

What this means for data leaders

Sangani and Mohan disagree about emphasis and agree about the shape of the problem. AI governance cannot live in one layer, whether that layer is the data, the context, or the agent. It has to follow a system that keeps changing, with rules that travel with the data, owners who keep the products alive, and versions that make corrections traceable. Leaders who build for that now will be able to swap models, add agents, and change interfaces without losing control of what the business means.

Curious to learn how Alation's data products marketplace can support your AI governance strategy? Book a demo with us today.

Sources & notes

Every external claim on this page is independently verifiable. The public sources are listed here.

1. Sanjeev Mohan, author of

   Designing the AI-Driven Data Foundations, headed Gartner's data and analytics research team before founding SanjMo in 2021. He states that even with highly governed data, AI can misbehave and give wrong answers, so outputs must be governed. Source: TechTarget, "Successful AI deployment rests on strong data foundation," 21 Sep 2026. https://www.techtarget.com/data-technologies/feature/Successful-AI-deployment-rests-on-strong-data-foundation

2. Alation's comparison of the semantic layer, ontology, and enterprise context layer. Source: Alation blog (publication date to be confirmed by editor). https://www.alation.com/blog/semantic-layer-vs-ontology-vs-enterprise-context-layer/

3. Alation's guide to governing agentic AI treats agents as governed runtimes: lineage on agent reads and writes, and policy enforcement at decision time. Source: Alation blog, 2 Feb 2026. https://www.alation.com/blog/governing-agentic-ai-trust-compliance-data-workflows/

4. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls. Source: Gartner press release, 25 Jun 2025. https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027

5. GPT-4 (March 2023) identified prime vs. composite numbers with 84% accuracy; GPT-4 (June 2023) scored 51% on the same questions. Source: Chen, Zaharia and Zou, "How Is ChatGPT's Behavior Changing Over Time?", Harvard Data Science Review, 12 Mar 2024 (Release 1). https://hdsr.mitpress.mit.edu/pub/y95zitmz/release/1

6. Alation Data Products ship with an owner, a versioned contract, quality enforcement, and lineage. Source: Alation. https://www.alation.com/aios/data-products/

7. The data product manager bridges business needs and data capabilities, pairs with a data steward, and ensures data reusability; the guide links reusability to avoiding redundant workflows and conflicting versions of the same dataset. Source: Alation blog, 21 Feb 2025. https://www.alation.com/blog/what-is-a-data-product-manager/

8. Claudeforce, the Salesforce and Anthropic partnership announced 26 Aug 2026, brings Salesforce data, workflows, business logic, and permissions into Claude; Salesforce states users get the same answer in Lightning, Slack, or Claude. Source: Salesforce. https://www.salesforce.com/claudeforce/

9. Episode: AI Radicals, "Metadata to Knowledge Graphs: Mapping the Full AI Data Stack, with Sanjeev Mohan, Principal of SanjMo." https://www.alation.com/podcast/episodes/metadata-knowledge-graphs-sanjeev-mohan/

ANALYST ATTRIBUTIONS & DISCLAIMERS (required only if Addition A1 is kept)

Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, press release, 25 June 2025.

Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. GARTNER and Magic Quadrant are registered trademarks and service marks of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved.