Rewriting the Governance Playbook for the Agentic Era

Key takeaways

• Erin McIntosh, VP of global data operations at CNA Insurance, traces her focus on governance to a working session with claims adjusters, where inconsistent reports meant each adjuster had to learn which version of the numbers to trust.

• CNA Insurance consolidated eight business intelligence tools into a single modernized global stack during McIntosh's first year leading global data operations, alongside new traceability and usage discipline for purchased third-party data.

• CNA Insurance is rebuilding data governance as an agent-led function, using agentic workflows

  to assist or replace human stewards, interpret regulations including the General Data Protection Regulation (GDPR), and serve quality insights out of the data catalog.

• McIntosh separates automating a decision from improving one: automation delivers efficiency only, while arming a decision-maker with better information is what changes financial outcomes.

• Prototyping cycles at CNA Insurance compressed from roughly three months to one to three days, which McIntosh says makes time-boxing and an agreed definition of done more important than they were before.

• McIntosh argues the real return on a data organization shows up as reduced cycle times, better underwriting decisions, and claims-experience efficiency, not as value attributed to the data team itself.

• CNA Insurance's next 18 months are focused on standing up a global governance body run agentically, with tighter coupling between data governance, AI governance, security, compliance, and regulatory functions.

Episode intro: Governing data at a global insurer in the agentic era

[00:00:00] Satyen Sangani: Welcome back to AI Radicals. Today's guest believes that the biggest challenge in enterprise AI isn't technology, but trust. Erin McIntosh is VP of Global Data Operations at CNA Insurance, leading a team spanning data governance, analytics, and third-party data strategy. We dig into why traditional governance models have struggled to scale, how agentic systems could transform stewardship and compliance, and why organizations need to stop optimizing for perfect pilots and start designing for production.

[00:00:31] Satyen Sangani: Erin also draws a sharp distinction between automating decisions and improving them, and why the latter is where the real business value lies. If you're trying to balance innovation with accountability and move AI beyond experimentation, this episode is for you

[00:00:48] Producer: This episode is brought to you by Alation. AI agents are only as good as the knowledge beneath them. Get it wrong and it shows up in production. Get it right with Alation at RevAlation. Coming to Chicago, London, and Sydney this fall, where data and AI leaders talk real measurable ROI. Save your seat at alation.com/revalation

[00:01:17] Satyen Sangani: Erin McIntosh is Vice President of Global Data Operations at CNA Insurance, one of the largest commercial property and casualty insurers in the United States. She leads a global team uniting business, technology, legal, risk leaders to transform data governance, building trusted systems, improving transparency, and ensuring responsible stewardship.

[00:01:36] Satyen Sangani: Her career spans more than twenty years across insurance and consulting with prior roles at Accenture, Nationwide, and Travelers. Erin, welcome to the show. 

[00:01:44] Erin McIntosh: Thanks so much, Satyen. Appreciate it. 

[00:01:46] Satyen Sangani: So I guess maybe to start, give us some context. Tell us about your work and what the current role is and your remit, and I guess we can go from there in order-- in, in terms of what you've done.

[00:01:58] Satyen Sangani: But, but let's just start on the overall context since that's the word of the day. 

Why do different teams get different numbers from the same data?

[00:02:02] Erin McIntosh: Absolutely. So where I'll start with is kinda how I entered into data and AI and kind of found my way into this space. And I really wanna call out, you know, um, after twenty-one years in the industry, there's a moment that really sticks with me, and I was sitting down with a variety of adjusters in the claim organization, and we were talking about processes that was going on, and they were talking about their different reports.

[00:02:24] Erin McIntosh: And what they found was that everybody kinda had some different interpretations, slightly different numbers, and it was that inconsistency that really struck. And each person had to learn which version that they wanted to trust and which one they wanted to use based on their own experience. And that became the system of finding the right pieces of information that helped their story, and that's really when it clicked for me that this isn't just a data problem, and it wasn't just a reporting problem, and it wasn't just a technology problem.

McIntosh's remit: Governance, analytics, and third-party data across four regions

[00:02:54] Erin McIntosh: It was a trust problem. And what I took away from that, and it's really shaped my career now here at CNA, is that once trust is gone, people don't stop or stop working. They really just kind of build their own version of reality, and that's the perspective that I'm bringing in to CNA. So my remit here at CNA is the global data operations lead.

[00:03:15] Erin McIntosh: I really sit at the intersection, as you mentioned, between governance, analytics, third-party data and at a global basis. So CNA has operations globally, essentially focused in the United Kingdom and Europe, Canada, and the United States, and my focus is really making that data usable. And help inform how businesses' decisions are done at scale.

[00:03:35] Satyen Sangani: And how, how long have you been in this role? 

[00:03:38] Erin McIntosh: Oh, we are a month shy of one year at CNA, so we're coming up on the year anniversary. It's been a fast and feverish pace that we've set. I- I've told often both my team and the organization, "Lace up your sneakers. We're gonna go at speed." And I will say, we are, we are moving very quickly.

[00:03:56] Satyen Sangani: Yeah, an apt analogy 'cause you're a sort of ultra distance runner, I think. So you, you started a year ago. Tell us what's happened in the year. 

A year of rapid transformation: Modernizing BI and third-party data at CNA

[00:04:04] Erin McIntosh: Well, I think like a lot of organizations, AI hit us pretty fast with going from ChatGPT, so I think that's, you know, one of the things. Coming into this role, it was really understanding with my role being global, and one of the first few roles that were being truly mandated on a global basis, we were working on how do we integrate governance with security and architecture.

[00:04:24] Erin McIntosh: In this year, we've delivered a full BI modernization front globally, so really taking... You know, we had eight tools across the organization, streaming that down to really use the right technology where it matters We've established some discipline around our third-party data. We spend a significant amount of investment in third-party data assets, getting that traceability, getting that usage down.

[00:04:45] Erin McIntosh: And now with AI, we're beginning to build an AI-enabled reference data platform, AI-enabled governance, and also how do we get an organization ready to actually consume AI products? So that's where the focus has been, I will say, in the last three to six months. 

[00:05:00] Satyen Sangani: So a little bit of focus on the tooling, a little bit of focus on understanding the, the shape of the data, as it were, and now as you're moving forward, you're looking a little bit more on AI-oriented tools for reference data and for data consumption writ large.

Why does process-based governance translate well to AI agents?

[00:05:14] Erin McIntosh: I think we're really trying to think about what is the future of governance, honestly. So governance historically was very much process based—that lends very well to AI or agentic-led governance, which is where we're headed—and we're thinking about not only how we can use agentic capabilities in the governance space, but also how do we prepare our data to be used across the organization as we're thinking about enhanced AI capabilities.

[00:05:37] Satyen Sangani: Got it. And you're using what tools in that particular world, and, like, what are you... and how much of this is are you building sort of internally, and how much of this are you building through third parties? 

Should you build your own AI capabilities or buy them?

[00:05:48] Erin McIntosh: So it's a combination, and I think that's really what we're trying to reimagine. If you asked me three months ago would we build certain capabilities ourselves, I would tell you absolutely not.

[00:05:58] Erin McIntosh: Go to the market and buy a solution. I think with different technologies, whether that be the CloudCodes of the world, whether that be OpenAI, et cetera, it's having us and getting us to pause and kind of think about, well, should we? Should we buy? Should we build? That has changed substantially, and I would say we're kind of write- rewriting the playbook right now on what does that look like.

[00:06:18] Erin McIntosh: For us, it is very much a partner, but also we are standing up pods to build these capabilities ourselves where it makes sense. 

[00:06:27] Satyen Sangani: So it's a blend of both. 

[00:06:28] Erin McIntosh: It has to be a blend of both, yeah. 

[00:06:30] Satyen Sangani: And when you think of data being more usable, how do you define that? Usable to who? How do you assess usability? How do you measure it?

What makes data usable for AI?

[00:06:37] Erin McIntosh: So I think it's... it comes down to a variety of different components, right? Accessibility is one. Is the data available to actually be consumed in the format that it's in? So if we talk about structured, unstructured data, how can it be pulled into the various capabilities, whether that be traditional reporting, traditional kind of predictive analytics, or now as we're thinking about AI.

[00:06:55] Erin McIntosh: So that's one thing is accessibility. The other gets into the trine- tenets of governance. Is the information accurate? Is it of quality? Is it timely? And also, do people trust the information? And that's really where we're going back to our basics and saying- Have we governed this information appropriately?

[00:07:11] Erin McIntosh: Do we understand the information that we are capturing, and should we be using it for those purposes? So it's, it's really having us take some steps back and saying, "Governance is really essential to how we move forward, and we have to put in some legwork." But with agentic capabilities, we're also seeing us being able to move a little bit faster and at scale in those governance components.

What makes data usable for AI?

[00:07:30] Satyen Sangani: Let's switch gears a little bit and talk about the business more broadly. So you've been working ... I know you have a couple of different partners, Alation, Accenture, I think Google, and, and you might name others. And one of the things you've been working on is AI-powered underwriting. Tell us a little bit about that use case, and tell us a little bit about sort of the data aspects of that use case and, and what, what you're working on and what problems you've been solving for and, and, and the, the texture of sort of what that is.

[00:07:56] Erin McIntosh: Yeah. I will say in terms of, you know, the challenges that we're facing, similar to probably a lot of insurance carriers, right? Between your claims, your underwriting, your operations, we're thinking about and trying to not just replicate or automate processes, but how do we reimagine them? In the underwriting space, we are looking at everything from submissions to pricing strategies, et cetera, so anything that would normally hit the underwriter's desk.

[00:08:21] Erin McIntosh: In the claims space, similarly, how can we do more and get insights to those individuals faster so they can take better information? It's not just about process automation. It really is that reimagination, and that's where we're spending time really looking critically at those processes. Do we understand the outcomes that we're searching for, and can we augment with technology?

[00:08:43] Satyen Sangani: What are the insights in this work that you've, you've found? I mean, so as you think about sort of process re-imagination, a lot of that is about sort of people reimagining their roles and how they do work. I imagine that's really quite complex. Tell us a little bit about that work and what you found and what the learnings have been.

How do you tell which process steps AI should eliminate?

[00:09:01] Erin McIntosh: So you're right, it is complex, and I think it's an understanding of how do we balance the technological components where a lot... like a lot of organizations, we put time and money into policy implementation or system implementation, claim system implementation. We are now looking critically at not just what the process is, Satyen, but what is the outcome that we're actually trying to achieve.

[00:09:25] Erin McIntosh: And that has gotten us to question, do you really have to go through all those steps that we historically were doing? And that has hit, honestly, not just underwriting, but I will tell you any one of our functions focusing on what is the outcome, why are we doing this step, and then tracing back through the process.

[00:09:40] Erin McIntosh: And it's got us to really re-rethink what work is required. Why do we do that? And it's gotten us to, I think, challenge ourselves. Even in the governance space, I will tell you, I'm challenging my team every day on why are we doing that stuff? What value does it produce? Does it help us in the outcome? If not, let's question that.

[00:10:00] Erin McIntosh: So that's going on a lot in our different spaces. 

[00:10:03] Satyen Sangani: You know, the thing about processes is the people who obviously do these processes, and they've built their jobs around these processes, and they know the processes inside and out. And of course, the big tension is that the pokes-- folks that are often doing the process can't necessarily look above it, but they have good...

[00:10:17] Satyen Sangani: they've got, of course, all of this implicit or explicit knowledge that they leverage in order to do a, uh, do and accomplish the process. How are you organized, and how do you sort of recognize when a process can be rethought and which parts of it can be sort of eliminated altogether versus the ones where...

[00:10:33] Satyen Sangani: How are you structured to make that, to make that, to recognize that? 

How do you structure teams to redesign a process with AI?

[00:10:36] Erin McIntosh: Some of the work that we're doing is actually embedding individuals in like a pod structure with the actual business partner. So this is not a technology going off and using the new tools and techniques to come back to the business.

[00:10:50] Erin McIntosh: It's actually bringing them along in that journey. So what we're doing is actually taking both technical and non-technical individuals as well as the business folks, determining what is the problem, having those individuals sit alongside the business folks to understand what are they actually trying to do and why.

[00:11:07] Erin McIntosh: And then developing very quickly solutions that we anticipate we could throw away, and that would be absolutely okay. And then we're also supplementing those teams if we do see some richness in those ideas with an implementation team. How do you harden it? And I think that's one thing that we are really trying to do, and I think organizations really need to think through, is testing and pilots are fantastic, and you do really want to explore these techniques and technologies.

[00:11:32] Erin McIntosh: You've got to get to a point where you're thinking, "Could this scale? Would this scale? How would I scale it?" So it's not that you're throwing all of this work away because the documentation, the consideration of the outcomes, all of those items is all rich goodness that you'll continue into the next phases.

[00:11:47] Erin McIntosh: So that's something that we're really trying to push on, is what the outcome is in mind so we can align the best approach to this. 

[00:11:54] Satyen Sangani: Yeah. Makes, makes complete sense. And, and so this idea of kind of prototyping- Yeah ... seeing what works, then hardening, that's got to be a new muscle, I would imagine. How are you learning that muscle?

[00:12:04] Satyen Sangani: And, like, talk a little bit about the transformation that you've, you've made there. 

Prototyping at CNA now takes days instead of months

[00:12:08] Erin McIntosh: I will say it's not a completely new muscle, but it's a muscle- Okay ... that we are trying to move with a pace that the organization has not seen before. So prototyping is not novel, right? This agile type development and co-location is not new.

[00:12:23] Erin McIntosh: The pace in which we are doing it, so I will tell you what used to take us three months is a day, two days, three days now. And this is where the volume of activity that can be produced is massive, is just really we're focusing on time boxing, being very intentional about what the definition of done can be for the organization before we get too far down.

[00:12:48] Erin McIntosh: Because again, this is what I think AI's really doing for organizations. It's allowing you to move with a sense of pace that we haven't seen in the past. You can get down a rabbit hole very quickly, and if you have incorrect information, you can get in a bad spot very quickly. So it's just being intentionally about that.

[00:13:04] Erin McIntosh: So the organization is, is working on that pace. I think the other thing is that these tools are allowing partners in our organization that have not traditionally been considered developers to actually develop and design, and that's been, I think, a culture change for the organization. What was traditionally a technology type role is now moving closer and closer into a more business centric or analytical role, which is different.

[00:13:31] Satyen Sangani: You talked a little bit about sort of, um, and this kind of process sort of speaks to it in terms of getting to understand the business process- Mm-hmm ... prototyping some stuff, seeing whether or not it sticks, and then scaling it. And so the- the... another way of viewing this is sort of the difference between automating a decision and improving a decision.

[00:13:46] Satyen Sangani: And so talk a little bit about that distinction that you've made. What does that, what does that feel? What does that look like? And why do you even make that distinction? 

What's the difference between automating a decision and improving one?

[00:13:52] Erin McIntosh: So I think automating a decision is really just how do we get as fast through that decision process as possible, right? What we're really trying to do is that improving the decision.

[00:14:03] Erin McIntosh: So can I bring additional information? Can I bring additional insights? Can I arm the individual with better information to make those decisions at scale? That is gonna lead to actually better outcomes, and this is where ROI really becomes into it, right? So if I can actually improve the outcome and improve not just the pace of the decision but the quality of that decision, then I'm gonna actually win in the marketplace, and this is something that we're really trying to drive.

[00:14:30] Erin McIntosh: And automation just gets you efficiency. That's all it's going to do for you. It doesn't actually improve the financial components to your organization. 

[00:14:37] Satyen Sangani: Do you ... So most of the AI work that you're doing, is it fair to say then that most of the AI work you're doing is basically quality slash quality improvement based more, more than it is actual efficiency oriented?

[00:14:49] Satyen Sangani: Or, or is it a blend of both? 

Should you start with efficiency or with better decisions?

[00:14:50] Erin McIntosh: I think it's a blend of both as people get more comfortable, right? So we start with usually efficiency because we know what the outcome should be. We want to see how much more quickly we can get through a process. And then efficiency can be with the ultimate decision. It also can be at the speed at which we develop.

[00:15:05] Erin McIntosh: As I'm looking at the overall governance, third party data evaluation, all of those lines, we're, we're truly trying to reimagine, and I ... when I say even reimagine, it's a little bit of a green space where these processes were defined but maybe not adhered to in the best way, and this is where we're really trying to rethink how we can do this, and what stops along that process can we actually drop out of it to make actually better informed decisions.

Are data teams being measured differently now that AI is here?

[00:15:31] Satyen Sangani: Yeah. It is interesting that, like, you, you mention you, you, you think about quality 'cause I think that's sort of the entire game with AI. Yeah. You know, on some level, AI's gonna give you something, and it's either gonna be, you know, it'll be directionally right, so probably more than 50%, but in some cases it might be just 50%, and in other cases it might be 99%.

[00:15:49] Satyen Sangani: And you know, for us, as we think about sort of data teams and getting answers out of it, it's really a system. Like, you think about getting agents to answer questions and then getting fidelity out of data sets, it really comes down to, okay, am I able to trust, to your point, this thing? Am I able to trust this system?

[00:16:05] Satyen Sangani: And quality is a huge part of the game and, and, and sort of making sure you have a t- clear return. And so this idea of sort of efficiency and quality seems to me to be the ... I guess maybe there's also throughput. Like, are you able to get more out of the systems? Yeah. You know, data teams have historically been not very measurable.

[00:16:21] Satyen Sangani: Yeah. Do you find that that's changing in the world of AI? Do you feel like that's a ... Is that a ... Are, are people trying to measure you? Are you trying to measure the team? Or, or, or are you, uh, are you not? Is that not a thing? 

How do you prove ROI on a data investment?

[00:16:32] Erin McIntosh: You know, it's one of those things that I think is very, very difficult. The value or the, the return on the investment in data projects in general is very difficult to attribute.

[00:16:40] Erin McIntosh: Yeah. Because you don't operate independently, right? Data for the sake of data doesn't actually solve a problem. You have to co-mingle that with whether it be your operations functions, your underwriting, et cetera. We are being asked, and I think we're trying to figure out, and it comes back to honestly one of the questions you were talking about, the build versus buy.

[00:16:58] Erin McIntosh: You know, what's the cost of AI, and is it gonna be cheaper and more efficient and more effective to do it this way even with that throughput and quality than having a human doing it? And kind of making those trade-off conversations. We're having a lot of those discussions. And also the cost of maintenance.

[00:17:13] Erin McIntosh: For us, in the terms of the data space, we are being asked in terms of what the value is. I think people are starting to be more confident and comfortable that data is a cost of doing business. You have to invest in it. The implications of poorly, poor quality data though, that's where I think we're trying to actually answer the question.

[00:17:32] Erin McIntosh: So what's the implications of not having good data? And I think the AI and the speed at which we're delivering has broader implications, and trying to quantify that is gonna be more critical for us. The other thing that I would say is when we talk about ROI, the real ROI doesn't show up in terms of, you know, how much value do your data organization deliver to you.

[00:17:52] Erin McIntosh: It, that's not really where it is. It's, you know, have we reduced cycle times? Have we improved quality underwriting decisioning? Have we created efficiencies and customer experience in the claims space? Like, that's what we're trying to really get to, and we've got to anchor around what are those true business outcomes, and knowing that data's an integral part to it, 'cause you cannot do those items without the data.

Matching the technique to the outcome: Efficiency, optimization, or better decisions

[00:18:14] Satyen Sangani: Yeah. So you have, you have direction. The outcome is sort of the direction. You know what the North Star is and where you wanna go. And then, and then I think what you're saying is, but we, we are not necessarily gonna impact the direction directly. We're a part of the system, but we want to basically figure out a way to influence that in, in productivity means or quality means or in so- in some cases, efficiency means, uh, amongst others.

[00:18:39] Erin McIntosh: Absolutely. It depends on what the outcome is trying to do, and we want to apply the appropriate techniques, the appropriate tools, the appropriate data where it makes sense to do that. And I think that's one thing, too. It's not just all... You don't use the same tool for every problem, so that's what we're really trying to partner with the organization to determine, is this an efficiency play?

[00:18:55] Erin McIntosh: Is this an optimization play? Is this just a better, improved decision that we want to provide? So that's what we're working and partnering very closely, and I think that's what you're seeing also, is that the data and analytic organizations becoming more and more a key component of the various business functions.

[00:19:14] Producer: Quick break, brought to you by Alation. Here's a stat that should stop you cold. MIT research found only 5% of enterprise AI pilots delivered ROI in production. The problem isn't your models, it's knowing where to aim them. We've put together a free guide for enterprise leaders. It's a framework for identifying which processes are ripe for AI agents, what green flags and red flags to look for, and how to build a knowledge foundation that makes AI trustworthy at scale.

[00:19:42] Producer: It's not theory. It's drawn from how Global 2000 companies are actually doing this. Download it free at alation.com/ai-guide.

Why does AI's fidelity come down to governed context?

[00:19:58] Satyen Sangani: So one of the things that you're obviously responsible for is data governance, and people, as you think about sort of this improvement to cycle times and this improvement to sort of data's impact, y- often what people are now saying about AI is that really the fidelity of AI comes down to context and governance, governed context.

[00:20:15] Satyen Sangani: Like, that seems to be sort of like conventional wisdom. Are you seeing governance viewed differently given the AI conversation today? Or, you know, governance historically has been viewed as being slow and methodical and plodding and, you know, consensus-driven and difficult and, like, all of those attributes.

[00:20:33] Satyen Sangani: And within CNA, is that viewpoint changing, or do you feel like it's kind of the same even though it's important and people recognize that it's important? 

[00:20:41] Erin McIntosh: The consensus is people conceptually know that governance is important and it's critical, and we're hearing a lot about contextualization, and that's how we're gonna get AI to work.

[00:20:50] Erin McIntosh: The biggest question that we're getting is, "Okay, you've talked about governance for a long time now. Nothing's really changed in the governance space. How's it gonna actually work, and how's it gonna scale, and how's it gonna improve?" Because governance organizations have failed along the way, right? We've not seen them scale well.

[00:21:08] Erin McIntosh: We all have those stories from our prior experience. And that adage that governance is slow and bureaucratic, I will say, yes, it has been. The difference now, and this is the d- direction that CNA is taking, is we know we need governance. So how can we actually take our own medicine, and can we actually apply agentic capabilities to deliver governance at scale, minimizing or guiding those stewards, those business owners, those quality assessments?

[00:21:40] Erin McIntosh: So it's not the human and process dependency that we saw in the past, but really trying to transform it, and this is where we're saying, "Let's actually take governance and be agentically-led in this space because we have had resistance to change, resistance to adoption, seeing it as an over some burdensome task.

[00:21:59] Erin McIntosh: Well, then let's throw out the playbook, and let's rewrite it, and let's do this agentically-led." And that's the direction that we're headed, and that's actually what I'm most excited about, is not having to hire consultants and stewards and having them be, you know, 20% of their capacity, but really giving them a cockpit, if you will, on how we can drive governance and improve this, not just for day-to-day traditional reporting, but really enabling the future, and that's where we're headed.

[00:22:24] Satyen Sangani: Tell us a bit about that. Like, where do you expect to start some of that, and what, what activities of governance do you think are most ripe for, for disruption through using agents and AI? 

Which parts of data governance are ripe for AI agents?

[00:22:33] Erin McIntosh: Yeah. Well, I would say governance technologies, and Alation is a fantastic partner o- of us. But as we think about this, and I think Alation is making a lot of different steps into starting to bring additional capabilities.

[00:22:44] Erin McIntosh: A lot of the technologies were very disparate, whether they be your lineage, your quality, your metadata, your compliance. It was all scattered technology, and you didn't have the ability to easily bring them together holistically to show what is the process of governance and those, those co- kinda cockpit, if you will.

[00:23:02] Erin McIntosh: What we're trying to do is how do we take our fantastic catalog, the lineage that we have, and actually present that in a way that stewards can actually see the information. So we're kind of trying to centralize. So if you think about MCU, right? We're taking all of these different subcomponents, bringing them together, and then making smart decisions on top of it.

[00:23:21] Erin McIntosh: So a couple places where we're going to be focusing, one is around stewardship and data ownership. In governance, getting stewards to buy in, getting them to want to stay in the roles, getting them want to actually, you know, develop rules and definitions, it's a hard job. And after two years, they typically, you know, move on to other things, so that sustainability.

[00:23:42] Erin McIntosh: So we're gonna be focusing on how do we create agents to help those stewards and if not, actually become the stewards. So how can we do that? That's a major piece. Another piece that we feel is ripe for agentic capabilities, I will tell you, is compliance. So we've got on all seen the various legislation documents, GDPR, Secrecy Acts.

[00:24:02] Erin McIntosh: How can we actually create agents both to interpret and help guide us in that? And then the last piece that we'll be doing is how do we actually engage with our catalog and serve up those insights and quality together to actually guide those stewardships, so kind of bringing those items. So those are the places that we're gonna start, but our intent is that while we'll have human in the loop, human on the loop, but eventually the agents will be driving a lot of our governance and being able to serve up strong insights that our other agents will be able to pick up in terms of we have an anomalies in the data, we have anomalies from a marketplace standpoint, and how should we be reacting.

[00:24:39] Erin McIntosh: So that's the intent that we're headed down. Um, it's exciting times. I think governance is an exciting place to be and for the criticality of not just where we are, but where we wanna go as an organization, I will say, and also an industry. 

Is it too risky to let AI agents run governance?

[00:24:51] Satyen Sangani: In rolling out these agentic capabilities and, you know, everybody talks about this idea of a d- AI data steward, and I think it seems like it's, you know, like a pretty obvious place to go, like manual work.

[00:25:00] Satyen Sangani: Nobody really wants to do it. A- and then of course, CNA is in the business of literally pricing risk. Yeah. And AI is stochastic, and therefore inherently within it, there's some risk. How does the organization view AI and its onboarding and, and how... You know, and even applying it in this relatively low risk domain, I guess, because you can patch it, but, but how...

[00:25:20] Satyen Sangani: Tell us a little bit about the, the onboarding process of AI into this fairly more manual and traditional process. 

[00:25:26] Erin McIntosh: It's a good topic 'cause this is where conversations that we're actively having, and the way that we've described it and really approached it, if you're not doing governance in all the various spaces that you should be or would want to be, it's basically a greenfield.

[00:25:39] Erin McIntosh: So the risk of doing something is better than not doing anything in this space. And I, I really hate saying that, but that's really where we're at. So it's a conversation that we'll have to have. It is internally focused. These are for our governance channels. We know we're not pushing this out, so we feel that it's a relatively safe place to start.

[00:26:00] Erin McIntosh: We will have more of a human direction in there to start with, and then we'll be able to toggle this off. Now, that's how I can speak to my governance space and how we're approaching it. 

[00:26:09] Satyen Sangani: Yeah. 

[00:26:10] Erin McIntosh: If other functions in the organization were being much more conservative, we are a very conservative industry, I will say, on how we apply these technologies, but I think governance is a really good place to start this and with a relatively low risk as we think about the application.

[00:26:27] Satyen Sangani: Yeah, I mean, I think this is kind of the interesting idea, which is that the applications like our stewardship or curation automation end up, like, being able to allow you to steward hundreds and thousands or millions of data elements in, in hours or minutes and, and that's amazing. And then, you know, there's this question of, okay, well, is it right?

[00:26:44] Satyen Sangani: But then to your point, it's kind of a function of abundance, which is if, if the answer is the null set or if the alternative is the null set, then it's, it's net, generally net better unless it's creating a whole bunch of noise. So now you've got all of this sort of... You've got this new prototyping going on.

How do you decide when a pilot is ready to scale?

[00:26:59] Satyen Sangani: When do you decide when you're doing the prototyping, when do-- what are the signals that you're looking at in order to be able to decide whether a prototype gets to production? Tell us a little bit about that process and, like, what are the things that have actually made it through, and when have you found signal versus deciding to say walk back from something?

[00:27:15] Erin McIntosh: So I think like with anything, whether it be AI or any one of our predictive solutions or prescriptive analytics, et cetera, it's when do you decide to actually implement at scale, whether you push that to production or more formally roll out. What we're looking for is around both the Targeted outcome, and this is something that we actually push on very heavily, of what are you trying to actually achieve as a result of this pilot?

[00:27:37] Erin McIntosh: So it's not a pilot for the pilot's sake. So what are the learnings that you're looking for? Whether that be we're looking for efficiency, we're looking for an improved outcome. We also look very clearly and ask for what is the control that you're comparing against. So oftentimes, I will say especially in when we're looking at third-party vendors right now in terms of data quality, that build versus buy decisioning, looking at what can we develop internally and what can we develop externally, and actually comparing the two together.

[00:28:03] Erin McIntosh: Once we see that and see those trends, then we'll determine what is the cost to go out and scale the solution and what would be the adoption that we would get. So it comes down to cost, it comes down to speed, and it comes down to the value that we are seeing, but also the perception of the business partners that are gonna use this.

[00:28:19] Erin McIntosh: Is there buy-in there, and are they willing to accept the technology and wanna use it? 

How do you build an agentically-led governance organization?

[00:28:24] Satyen Sangani: So tell us about the road ahead. So lots of work- Yeah ... obviously done and in the process of being done. Where do you see the program going over the next three to five years? What's your vision for, for the, the program, and, and what...

[00:28:36] Satyen Sangani: You know, if you were to paint the vision of what the future looks like, what does that look like? 

[00:28:40] Erin McIntosh: I will not go out five years. So three is probably you're gonna... The longest you'll ever get from me, maybe even two. Yeah, seriously.

[00:28:45] Satyen Sangani: In this day and age, even, like, five months is hard, but- 

[00:28:47] Erin McIntosh: It... And, and that's, I think, one of the biggest challenges right now, is that you feel that you're building a foundation a little bit on quicksand because it's moving, right?

[00:28:54] Erin McIntosh: And, or I should say like a dust storm, right? It's shifting on us a little bit month to month, quarter to quarter. What I would say what the future looks like for at least my organization as we think about governance, we think about third-party data, and we think about kind of change and adoption and those type of topics, it will be agentically centered.

[00:29:11] Erin McIntosh: I will tell you that straight out that that's where we're headed and how we do this. So agentically led, we will be more of a global organization, meaning how do we share insights across the organization, both within our core kind of foundational functions, but also within the business units. There'll be stronger partnership between security, compliance, regulatory, and data governance as well as AI governance.

[00:29:36] Erin McIntosh: We need to be a very tight-oiled machine looking through these as we're thinking about regulatory and legislative decisioning, AI utilization, the data products that we're gonna use. So we have to become much, much more efficient because the pace at which our business wants to move is gonna demand this to be so.

[00:29:54] Erin McIntosh: I do think that the volume of data that we are consuming and also producing for the organization will change to be much more higher quality. So we talked about big data for a while. We were talking about the kind of volumes of information. It's really how do we synthesize that down to get that we can actually make action on because honestly, insights are great, and we talked about data is really good and information.

[00:30:16] Erin McIntosh: It's gonna be what actions can we make and how fast can we make those with the right confidence. So that's where we're gonna be heading. I would say the next 18 months is all about getting our governance body stood up globally, having that in place done agentically, and then we will see what else the future has to hold for us.

Do you still need to review the code an agent writes?

[00:30:32] Satyen Sangani: One of the things that I think a lot about are in this world of kind of knowledge perfection and quality as feedback loops. You know, we, we think that basically all of this, all of this boils down to some combination of agents improving or reading context, acting on data, errors being discovered, and then basically figuring out what to do with the errors or what to do with the issues, right?

[00:30:55] Satyen Sangani: And, and so- Yep ... this kind of all... It's not that the work disappears, but the work actually shifts. And, you know, you see this in code review. The- Yep ... in, in the world of code review, you know, just having a conversation with another customer and they were like, "You know, we've got all of these engineers reviewing all of this code.

[00:31:09] Satyen Sangani: There's so much code, and we can't even figure out how to review all this code," and now they build agents to basically review the code. And so you think about the kind of this feedback loop work. I guess, have you started seeing these loops? Does that resonate as a comment? And what, can you talk about some of the use cases where these things have come up to the extent that they have?

[00:31:24] Erin McIntosh: Yeah.

[00:31:25] Erin McIntosh: Where I'm seeing them come up directly is in our actual BI modernization effort, where we're looking at semantic models and the consumption of our data products and how do we actually create those feedback loops, not only to review code But again, reimagine the process. Can they actually write the code to the processes and the standards so then we don't actually have to review the code?

[00:31:44] Erin McIntosh: So we're seeing it directly where people are starting to ask the question, "Well, could I use it to review data product development, BI reports, et cetera?" You could, but why not just have it develop it for you the right way from the beginning? So we're kind of actually trying to bust some of those, those processes.

[00:31:59] Erin McIntosh: The other places that we're seeing it apply to is definitely Just overall process optimization as we think about our filings, as we think about how we're actually, um, our code appetite guides and a lot of other knowledge capture information. How are we actually serving that up, and how do we create feedback loops where it's not just doing a sampling like we normally would do, but actually reviewing all of our files, all of our claims, all of those for more of that code insight and quality insight.

Will AI replace dashboards and BI reports?

[00:32:29] Satyen Sangani: Yeah. It's great that you mention this con- the, the BI dashboard project where you've sort of rationalized from kind of these eight tools. Yeah. A lot of people are talking about sort of the future of BI, and you're obviously not necessarily one who's kind of building these tools, but you're obviously consuming many of them.

[00:32:44] Satyen Sangani: I don't know how much you've done with the Claude dashboards that exist out there, but it seems like they're certainly super easy to use. How do you see the world of that producing? I was talking to one customer, and they were saying to me, "You know, we are... envision a world where people create zero net new dashboards."

[00:32:57] Satyen Sangani: What do you see having, uh, studied sort of that world and having rationalized amongst a very confusing set of capabilities? 

[00:33:04] Erin McIntosh: I think the world of business intelligence consumption will change. I do feel that we will still, at least in the insurance industry, I do feel that you will need some sort of structured reporting for the purposes of kind of our standardized monthly packages, quarterly packages, right?

[00:33:21] Erin McIntosh: We have to have some sort of pixel perfect information, just the way that we operate and really defined. In terms of visualizations and dashboards and the ad hoc analysis, absolutely. I think that changes significantly whether the traditional tools play into that, whether it be Claude, whether it be others.

[00:33:40] Erin McIntosh: I think the whole idea of natural language querying, insight generation, um, graphical recommendations, I think it's gonna be a huge opportunity for the organization. We are piloting those as we speak. So we are starting in leveraging some of the co-pilot technologies in terms of development of dashboards.

[00:34:00] Erin McIntosh: So actually taking individuals that had not historically been exposed to these tools, had not historically developed, letting them use the agents to actually develop their dashboards. We will get into AI for BI in the next quarter, and this is where we're very excited as well on what does that really look like in terms of behaviors.

[00:34:19] Erin McIntosh: I will say, organizations move a little bit slower in terms of the visualizations and dashboards. I, I recall a time where we thought PowerPoint packages were gonna be dead, and you would use everything in terms of visualization, whether it be, you know, Power BI, Tableau, et cetera. That did not happen. So it'll be interesting to see- Okay

[00:34:36] Erin McIntosh: if it is adopted. I would love to get away from packs. I would love to have more of that interactive type discussion. It's just gonna be how do we kinda change the culture to be very comfortable, and it all comes down, Satyen, to do you trust, and is the data behind those tools consistent? How you're interacting, and do we get deterministic responses?

[00:34:56] Erin McIntosh: From the tools. So we got a, we got a long way to go, but I think it's gonna be very exciting on how that works. 

Quick hits: AI's biggest misconceptions, wasted effort, and governance myths

[00:35:00] Satyen Sangani: Well, let's switch gears a little bit. We have a quick hit section, so we're just gonna jump into 'em, and we're gonna go quick answer, and you can disagree with the premise or you can answer the question.

[00:35:11] Satyen Sangani: Okay. Uh, what's the most dangerous misconception about AI and enterprise data right now? 

[00:35:16] Erin McIntosh: That it's a technology. 

[00:35:17] Satyen Sangani: Mm. What should companies stop doing immediately? 

[00:35:21] Erin McIntosh: Focusing on a pilot as opposed to thinking about how they can actually scale a solution. 

[00:35:26] Satyen Sangani: Ah, nice. Where is the biggest waste of time and money in AI deployments right now?

[00:35:32] Satyen Sangani: Follows from the last one probably. 

[00:35:33] Erin McIntosh: I think seeking perfection as opposed to progress. So especially as we think about all the things that could go wrong with this and the, the fear and trepidation, I think we need to focus really on how do we establish the right use cases to test the functionality and get us to move forward.

[00:35:48] Satyen Sangani: Okay. And let's go back to data governance 'cause that was, that was one that was changing. What's the one belief about data governance that's just wrong? 

[00:35:56] Erin McIntosh: That it is slow, and I will say good data governance is actually effective. Bad data governance is actually slow and burdensome. 

[00:36:05] Satyen Sangani: Brilliant. Erin, it's been amazing having you on.

[00:36:08] Satyen Sangani: The perspective and the, the work and just sort of the thoughtfulness is, is super apparent. We'll probably have to have you back on to see what you end up doing over the next year, but we thank you for the time, and I know everybody enjoyed ... I was gonna enjoy listening and I absolutely enjoyed having the conversation.

[00:36:21] Satyen Sangani: Thank you. 

[00:36:22] Erin McIntosh: Thanks so much, Satyen. Appreciate it. 

Sangani's takeaways: Trust as the limiting factor in enterprise AI

[00:36:26] Satyen Sangani: Erin's core belief is one that I keep coming back to. The biggest challenge in AI isn't technology, it's trust. For years, governance has been viewed as a slow, manual, and bureaucratic process. Erin sees AI as a chance to rewrite that entirely, using agents to scale data management so governance becomes an accelerator for innovation, not a barrier to it, and that connects to her sharper point.

[00:36:47] Satyen Sangani: The goal isn't to automate decisions, it's to improve them. Efficiency alone doesn't create a competitive advantage. Better decisions do. The organizations that win in the AI era will be the ones that build trusted data foundations on the way to solving business problems, not the ones that focus on capabilities alone.

[00:37:04] Satyen Sangani: I'm Satyen Sangani, CEO of Alation. Thanks for tuning into AI Radicals, and see you next time 

[00:37:11] Producer: Before you go, a word from our sponsor, Alation. You've been listening to us talk about making AI work in the enterprise, and here's one of the biggest unlocks. Your data has to be trustworthy before AI can be.

[00:37:24] Producer: That's exactly what Alation's Data Products Marketplace is built for. Data teams use it to package raw data into certified, governed, reusable data products without writing a line of code. An AI-assisted builder does the heavy lifting, governance travels with every product automatically, and business users can chat with data in plain English and get accurate, explainable answers.

[00:37:47] Producer: Companies like Cisco, Pfizer, and NASDAQ are already doing this at scale. If you want your AI initiatives to actually land, this is where it starts. Download the free data sheet and see what AI-ready data actually looks like at alation.com/data-products.