Making Agents Work: Why "Fix the Data First" Is the Wrong Place to Start

Quotations are transcribed from the session at Imagination In Action, Google Bay View, 14 September 2026.

Ask a data leader what stands between an agent pilot and production… and you may hear, “the catalog.” It's incomplete. The definitions are stale. The lineage has gaps. Fix that first, then ship.

The gap that instinct is trying to close is real and measurable. Deloitte's 2026 Tech Trends found 38% of organizations piloting agentic solutions but only 11% running them in production.¹ Something is stopping the other 27% from crossing over, and it is rarely the model.

But the instinct to go fix the catalog and come back later gets the sequence backwards, and a panel at Imagination In Action's summit at Google Bay View spent an hour explaining why. The session, called Engineering the Agentic Enterprise,² was handed an unusual brief: talk about the underlying, unglamorous things most people don't talk about once these systems start to scale. Melissa Valentine, Associate Professor of Management Science & Engineering at Stanford and a Senior Fellow at Stanford HAI, who studies how AI gets implemented inside organizations,³ joined Liran Zvibel, co-founder and CEO of WEKA,⁴ and Satyen Sangani, co-founder and CEO of Alation, with Ashish Verma, Deloitte's US Chief Data and Analytics Officer, moderating.

What emerged was a different definition of agent-ready. Not a catalog you finish, but a set of practices that hold up while the business moves underneath them: context tied to a working use case, a decision about where truth lives when something breaks, verified human ground truth under the evals, people who can hold both business meaning and technical mechanics at once, and credentials that survive every nested call. Let’s dive in!

The catalog reflects your reality

Sangani pushes back when customers tell him their catalog is a mess and agents can't understand it. A data catalog describes what is. "The catalog reflects your reality. If your map has spaghetti all over the place, your map is just going to show that fundamental reality of your systems," he said.

The complaint, in other words, is usually not about the catalog. It is about the estate the catalog is faithfully describing. Cleaning up the map does not clean up the territory, and a catalog that has been groomed to look tidier than the systems underneath it is less useful to an agent. Today, companies preach context as a sure path to AI accuracy. But, as Sangani pointed out, "Context in isolation is a silly thing to go after because how do you know you're done, or where to get started?" There is no completion criterion. Documentation projects launched without one tend to expand until the budget runs out, and the organization ends up with more written material and no working agent.

The only workable definition of enough context is an application that performs, and keeps performing as the business changes underneath it. Start from the use case, or you are accumulating documentation. That is the sense in which "agent-ready" is a property of a deployment rather than a state of a repository.

Context drifts, and someone has to decide where the truth goes

Verma put the architectural question to Sangani: AI runs on a substrate, hardware up, so given that grounding, what do organizations most underestimate once they start hitting the shortcomings of a deployment?

Sangani went to the property that separates agents from conventional software. In a normal application, you test, ship, and the edge cases you covered keep behaving. But, "Because these are stochastic apps, there's consistent drift." The underlying schema drifts. Business rules and context drift. Tools that were once appropriate begin to hang, perhaps because they are a version behind.

This is the deeper reason "fix the catalog first" fails as a plan. Even a genuinely complete catalog starts decaying the moment you ship, because the systems and definitions it describes keep moving. Readiness is not a milestone you clear once.

Evals are the minimum condition. What happens after a failure is the harder discipline, and Sangani's diagnostic is useful: Triage produces a new eval, and then the eval itself has to be made to pass, which means deciding what was actually broken. "Do I fix the knowledge base? Do I fix the underlying data? Was it a problem with data quality, access, or the agent prompt? Where does the truth go?" Each of those answers implies a different owner, a different system, and a different repair. Triage in an agentic system is an ongoing governance function, not a one-off support ticket.

And once you locate where truth goes, a harder question follows: "If I change the truth, is this globally true, or only true for this particular use case?" Sangani called these knowledge management decisions, "about what container you put these in and how you fix them," and warned that they scale badly: "As you deploy AI more and more, that complexity increases even more." Sorting them out depends on knowing your critical data elements and tracing them through data lineage.

The context that doesn't travel between agents

Zvibel described what evals now catch, which is different from what they caught a year ago. "A year ago, everyone was obsessed with hallucinations," he said. Grounded generation has improved sharply, though hallucination is far from solved: Stanford HAI's 2026 AI Index found hallucination rates ranging from 22% to 94% across 26 leading models on a factual-recall benchmark.⁵ What Zvibel is describing is narrower: "You rarely get the model to invent things, essentially because it now starts with a lot of context at the beginning if set up correctly." The industry solved a context problem… and in doing so surfaced a different one.

The failure that replaced it is structural, and it appears as soon as architectures get recursive. "When agents become nested, when they call each other, turtles all the way down, the problem many times is you have a nested agent that has a completely different context," Zvibel said. The nested agent answers the problem with a different set of understanding than the agent that called it. Each step holds up on its own. The chain does not. "You get something that, if you only employ logic, works out, but when a human looks at it at the end and everything needs to connect, it's completely ridiculous."

His remedy is procedural rather than technological. "So many times when you're deploying a new workflow, it's really important to go and verify that all the stages see the right data to make sense of the results." 

Turtles all the way down: The human layer under the evals

Evals, hands down. That was Valentine's answer when Verma asked what organizations most underestimate in taking an agent from pilot to production — the same gap between piloting and production that the Deloitte figures describe.

She had raised a case from her fieldwork earlier, during the opening round. Valentine described a financial services institution running its most mature agentic use case at roughly 99% resolution and unable to close the last 1%, because "they don't have the workforce expertise to go chase out that 1% of exceptions, which are these gnarly edge cases that nobody really knows what to do with." The constraint here is human expertise. Those exceptions sit exactly where the organization's own understanding runs out — which is precisely the territory no catalog can document, because nobody has decided what the right answer is yet.

The surprise, she said, comes from an assumption companies carry in at the start. "Oftentimes there's this belief that you can have agents check agents check agents, and you have all these loops where agents are checking agents." That architecture can be powerful. It also has a floor. "At the bottom of all those loops of agents checking agents, you have to have a really nice layer of verified, typically human-created data for evals."

Loops of agents, and human judgment underneath holding it up: It's turtles all the way down.

Valentine was candid about why organizations skip that bottom layer. "It's really hard to create evals; it's really hard to define success in a way that humans can then apply it well enough so that you have this nice ground-truth data… They're hard to define, hard to do, and people hate doing it." Ground truth for AI governance is slow, unglamorous human work, and this tedium is the first thing an ambitious roadmap defers. 

Birds, frogs, and who owns the definitions

Someone has to write those evals, and on the question of who, Valentine has data rather than intuition. Five years ago she built a Stanford class called Managing Data Science Organizations for Innovation and Impact, covering the research on traditional machine learning in organizations. Across every paper she reviewed, one finding dominated. "The number one predictor of an effective ML implementation was hands-down the relationship between the data scientists and the domain experts. It wasn't even a subtle effect across all those studies."

She was curious whether that relationship would hold once generative AI arrived. For a couple of years she did not see much of it, because domain experts were mostly getting licenses rather than building bespoke use cases. What she sees now is the rise of forward-deployed engineers: "FDEs who have the technical expertise to build the system have to go talk to domain experts; they have to have enough business context to build the thing."

Cross-functional collaboration, in other words, remains the precondition for a successful working system, and it was the theme that ran through the whole hour. "If that collaboration doesn't work, your system is not going to work," she said. Ownership matters less than the working relationship between the two sides. "It's an organizational challenge because collaboration between the two is what matters most. Someone needs to own it, but they just need to collaborate really well across those groups."

Sangani was asked the related question directly: most catalogs are inventories rather than usable context, so what has to change to make one agent-ready, and who owns the definitions? He started with competence rather than the org chart. "Ultimately, the definitions have to be owned by the people who have the greatest level of understanding." In practice, that means several people at once. Technical roles are needed for technical system understanding, business roles for business process understanding, and what he called quasi-functional roles for understanding the systems that produce the information. End-to-end process ownership in an agentic world "requires a technical individual, a functional individual, and someone who owns the content." Three people or thirty, depending on the process and its sophistication.

Which raises the question of who can hold more than one of those perspectives at a time. Freeman Dyson's framing is useful here, and it surfaced in an earlier Data Radicals conversation with David Epstein about the limits of specialization.⁶ As Epstein renders it, a healthy ecosystem needs both birds and frogs: Frogs work down in the mud with the granular detail, while birds fly above them, integrating what the frogs find. Dyson's own 2009 essay put it in terms of mathematicians: frogs live in the mud below and see the flowers nearby, while birds survey the landscape from above.⁷

Epstein's larger argument is about range, the compounding value of people who can traverse specialties and carry general knowledge alongside technical depth. Agentic work rewards that profile, because somebody has to hold the business meaning of a correct answer and the mechanics of the chain that produced it in view at once. An expert that possess both frog-like technical acumen and a bird’s-eye business view is impossible to find, making teamwork essential.

An eval layer staffed by frogs alone will validate each step; an eval layer staffed by birds alone will validate the story. Neither one catches the locally logical, globally absurd output Zvibel described. That is what makes trusted agent output an organizational problem before it is a technical one — and why agent-readiness is not something a catalog can confer by itself.

The future of AI: Declared intent

Sangani closed his remarks by reflecting on where enterprise AI will ultimately go. Knowledge artifacts, whether catalogs, ontologies, or compute choices, "are secondary to intent." The promise of agents is that you declare what you want and it just happens.

Fixing the catalog first mistakes the artifact for the outcome. A catalog, an ontology, a context layer all earn their keep in service of an application someone is trying to make work, and that application is the only reliable signal that you have built enough of them. Between a declared intent and a result anyone would act on sits everything the panel kept returning to: evals with verified human judgment underneath them, birds and frogs arguing over what a correct answer looks like, a decision about where truth lives once the schema drifts, credentials that survive every nested call. None of it glamorous, all of it critical.

Curious how Alation can support your agentic AI strategy? Book a demo with us today.

Sources & notes

Quotations are transcribed from the session at Imagination In Action, Google Bay View, 14 September 2026. Every external claim on this page is independently verifiable; the public sources are listed here.

1. 38% of surveyed organizations piloting agentic solutions; 14% with solutions ready to deploy; 11% in production. Figures restate Deloitte's 2025 Emerging Technology Trends study, noted in the source as publication in process. — Deloitte Insights, "The agentic reality check: Preparing for a silicon-based workforce," 10 December 2025 ↗ https://www.deloitte.com/us/en/insights/topics/technology-management/tech-trends/2026/agentic-ai-strategy.html

2. Session "Engineering the Agentic Enterprise," Imagination In Action, Google Bay View, 14 September 2026, 10:30 AM. — Imagination In Action ↗ https://imaginationinaction.co/2609sv/14

3. Melissa Valentine, Associate Professor of Management Science and Engineering, Stanford University; Senior Fellow, Stanford Institute for Human-Centered AI. — Stanford Profiles ↗ https://profiles.stanford.edu/melissa-valentine

4. Liran Zvibel, co-founder and CEO, WEKA. — Forbes Technology Council ↗ https://councils.forbes.com/profile/Liran-Zvibel-Co-Founder-CEO-WEKA/9581b3e4-5101-4bad-873b-e427420e6808

5. Hallucination rates ranging from 22% to 94% across 26 leading models on a factual-recall benchmark. — Stanford HAI, 2026 AI Index, Responsible AI chapter ↗ https://hai.stanford.edu/ai-index/2026-ai-index-report/responsible-ai

6. David Epstein on birds and frogs. — Data Radicals, Season 1 Episode 5, "The Reign of Specialists is Over," Alation ↗ https://www.alation.com/podcast/episodes/reign-of-specialists-over-david-epstein/

7. Freeman Dyson, "Birds and Frogs," Notices of the American Mathematical Society 56, no. 2 (February 2009): 212 — written version of his AMS Einstein Lecture. ↗ https://www.ams.org/notices/200902/rtx090200212p.pdf