Mark T. Nelson On Why AI Hasn't Changed The Fundamentals Of Building Great Software

If code is nearly free to generate, why are so many AI products still mediocre? That's the question at the center of a conversation between host Satyen Sangani and Mark Nelson, Venture Partner at Madrona and former CEO of Tableau,¹ on the season four premiere of AI Radicals.

Nelson has spent three decades inside enterprise software's biggest platform shifts, from databases and middleware at Informix and Oracle, to cloud and SaaS at Concur and SAP, to analytics at the helm of Tableau.¹ Today, as a venture investor, he sits across the table from founders trying to figure out what's actually durable about this AI moment versus what's just noise. His answer: less has changed than the hype suggests, and the parts that have changed are more subtle than "software gets built faster now."

Why this platform shift feels different, even if the fundamentals aren't

Nelson doesn't dispute that AI is a genuine platform shift on the scale of mobile or cloud. What's different, he argues, is the speed at which it arrived and the strange nature of the thing itself. Cloud and SaaS were still, at their core, new ways to digitize business processes. AI is something else.

"It is not just, oh, there's a new way to build or deliver software," Nelson says. "It is a new thing in town, and I use the word 'thing' very intentionally... AI is this fascinating, encapsulated knowledge, non-deterministic kind of human attributes."

That novelty is also why Nelson resists the more extreme predictions about AI replacing human judgment wholesale. Pressed by Sangani on where he lands relative to forecasts of mass job displacement, Nelson draws a sharp line between knowledge and understanding:

"If you imagine the current state of the technology and what LLMs have brought us, it's not intelligence," he says. "These things still don't understand the world. They don't have judgment... It is the most amazing encapsulation of knowledge that we've ever seen. You now, essentially, have everything on the internet in your hand. It is still fundamentally a parrot. Having an actual conceptual understanding of the world and, really, judgment, is still missing."

He illustrates the point with a metaphor borrowed from computer scientist Yejin Choi's TED Talk, "Why AI is incredibly smart and shockingly stupid":² skyscrapers were a genuine breakthrough that reshaped cities, but no amount of skyscraper-building gets you to the moon. Large language models, in his telling, are an extraordinary skyscraper. But they are not, on their own, a different category of machine.

The bottleneck didn't disappear; it moved

Writing code used to be the constraint on building software. That constraint has essentially vanished. What hasn't vanished is the harder work of knowing what to build in the first place, a distinction that matters a great deal for anyone thinking about AI governance and where trustworthy AI systems actually create value.

"We've never seen anything change the way we build software so fast in such a short amount of time," Nelson says. "It used to be getting the code generated was the blocker. And now, overnight, generating code is not the blocker. But great software isn't just flowing off of people's fingers, either. Software is flowing off of people's fingers, great software is not."

The skill that remains scarce, he argues, is what he repeatedly calls "taste": the judgment to understand a customer's actual pain point and translate it into something worth paying for.

"That bottleneck has just changed back to: do you understand your customer's pain point? Can you build a real solution for them?" Nelson says. "You've got a friend right in your pocket that can generate code, but do you have the good taste to understand what to build? Can you figure out how to sell it? Can you put all the things around it to make it a useful product for your customer?"

What VCs look for in founders now

Sangani asked Nelson directly what it takes for a new AI company to get funded today, especially as questions swirl about whether foundation model providers will simply absorb every promising use case: "Are the foundation models gonna do this? Is this in the blast radius? How do we build competitive differentiation? What does it even mean to have a software moat?" Sangani asked.

Nelson's answer starts with people, not technology. Madrona's three screening questions, he says, are consistent regardless of the AI cycle: why this problem, why now, why you. But the "why you" question has taken on new dimensions in an era where writing code is no longer a differentiator on its own.

"We all come with towering strengths and our own weaknesses," Nelson says, describing the founders he backs. "Not just being a product person, not just being an engineer, not just being a salesperson, all of those skill sets." The constant across every founder he trusts, he says, is a first-hand understanding of the customer's problem: "Do you understand your customer? Do you understand what you're solving and why? Do you really kind of first personally feel that pain?"

Enterprise buyers are done with AI for its own sake

Nelson's read on the buy side echoes his read on the build side. The earlier phase of AI adoption, where budget existed simply because AI was mandatory, has ended. He points to a former Tableau marketing colleague, now a CMO at a small startup, who described running a team of 18 people with 35 different AI tools already purchased. Her own assessment: that number of tools is not sustainable, but experimentation was necessary because nobody yet knew what would actually work.

That shift from experimentation to expectation shows up in how Nelson talks about pricing. He raises a question he expects more enterprises to ask as AI budgets come under scrutiny: will billing models built around usage hold up once buyers want a clearer line to ROI? It's an open question the industry is actively working through, and one where reasonable people land in different places.

"What's different this time about AI compared to the other big transitions? The staggering amount of capital," Nelson says. "Holy cow, are we spending a lot of money? People are going, but what am I getting for all this money? Am I really getting the return for it?"

He acknowledges the logic hyperscalers use to justify enormous capital outlays, while suggesting it can't hold indefinitely: "The hyperscalers, in particular, have been very open on the opportunity of cost of missing this moment is more expensive to me than spending hundreds of billions of dollars," he says. "But that's got to become real. That is the reckoning that'll come sooner rather than later."

Categories are dissolving, and that's an opportunity

Perhaps the most useful frame Nelson offers investors and operators alike is his view that enterprise software categories, CRM, ERP, and the rest, are being pulled apart and reassembled in real time. 

He's watched this cycle before: monolithic on-premise systems fractured into best-of-breed categories during the SaaS era, then re-consolidated into platforms. AI is triggering another fracture, and Nelson doesn't think the winners are predetermined. Established players carry real advantages in data governance, security, and customer trust, but they also carry the innovator's dilemma of not wanting to cannibalize what already works. 

Startups carry the opposite trade-off: freedom to rebuild the rules, but no customer base or data to start from. Execution, not incumbency, is what Nelson expects to decide the winners.

The takeaway for data and AI leaders

Nelson's central message is a useful corrective for any team currently deciding where to place its AI bets: the technology has changed how fast software gets built, not why customers pay for it. Judgment, customer understanding, and disciplined product taste remain the scarcest resources, and pricing models built on tokens rather than value are unlikely to survive the coming reckoning.

Curious to learn how Alation's platform can help your organization pair AI investment with real customer and business value? Book a demo with us today.

Sources & notes

Every external claim in this article is independently verifiable. The public sources are listed here.

1. Mark Nelson's title (Venture Partner, Madrona), tenure as President and CEO of Tableau, and prior career at Informix, Oracle, Concur, and SAP Cloud Business Group. — Madrona ↗ https://www.madrona.com/team-profiles/mark-nelson/

2. Yejin Choi, "Why AI is incredibly smart and shockingly stupid," TED2023, April 2023. — TED ↗ https://www.ted.com/talks/yejin_choi_why_ai_is_incredibly_smart_and_shockingly_stupid

All quotes attributed to Mark Nelson and Satyen Sangani are drawn directly from the AI Radicals season four premiere episode transcript.