Episode transcript
Note: This transcript has been edited for clarity and readability while staying true to the spirit of the conversation.
Hello and welcome to Judgement Calls: Leading in the AI era.
I’m Carl Raath, and this is the weekly podcast for leaders who want to elevate their own impact and the impact of the people they lead.
Here’s this week’s idea.
When an AI pilot doesn’t deliver, the post-mortem almost always starts in the wrong place.
The tool gets blamed. The vendor gets blamed. Someone says the model wasn’t ready.
What rarely gets asked is the question that would actually explain the result:
Whose name was on the outcome?
You’ve probably seen the number that kicked off a lot of these conversations. MIT’s Project NANDA published a preliminary finding in 2025 that roughly 95% of enterprise GenAI pilots showed no measurable P&L return six months after the pilot.
That figure travelled fast.
It usually travelled without its caveats, so let me put them back.
It measured return within a very short window. The researchers pointed to poor workflow integration, limited learning and implementation gaps—not simply the underlying technology—as central reasons pilots stalled.
That’s a long way from saying AI doesn’t work. It says most organisations couldn’t demonstrate that it did.
Those are very different problems with very different owners.
Think about what has to be true for a pilot to show a measurable P&L return within six months.
Someone had to define what “return” meant before the money moved.
Someone had to capture a baseline so there was something to compare against.
Someone had to have the authority to change how the work gets done, because the return lives in the workflow more than it lives in the software.
If none of that existed, the pilot was structurally incapable of showing a return—no matter how good it was.
You didn’t run an experiment that failed. You ran an experiment that couldn’t succeed on paper, then read the blank page as a verdict on the technology.
That’s a failure of ownership before it’s a failure of software.
Ownership, in my view, is exactly where boards should be looking.
A Director’s Guide to AI Governance from the Australian Institute of Company Directors is clear on that point: boards are ultimately accountable for the adoption, implementation and outcomes of AI technologies in their organisations.
That accountability can’t be handed to a tool, a vendor or a program of work.
I understand why that lands as compliance language for a lot of directors—just another obligation on a long list.
I’d read it the other way.
It’s a leadership statement more than a compliance one. It says the results of your AI investment belong with the same people who own every other result in the business.
When I say it out loud, that shouldn’t be controversial at all.
It’s only the novelty of the technology that has made it feel optional.
Picture the standard pattern, because it repeats everywhere.
A vendor demo lands well. IT stands up a pilot. A few enthusiastic teams get licences.
Six months later, there’s usage data, a good demonstration for the executive team and genuinely happy adopters.
There’s also nothing anybody can point to in the P&L.
The review meeting asks why the tool underperformed, but nobody ever agreed on what “performing” meant.
There’s no named owner, no baseline and no decision date.
The sponsor was a function. The success criteria were vibes. The vendor’s own dashboard was the measurement plan.
Asking why that pilot didn’t show a return is like asking why a boat didn’t arrive when nobody chose a destination.
Now run that same approach past any other capital decision and notice how absurd it looks.
You wouldn’t approve an acquisition without a named executive owning the integration and the value case.
You wouldn’t fund a new production line without somebody accountable for what it produces and by when.
Somehow, AI spending has been granted an exemption.
I suspect that’s because pilots feel like experiments, and experiments feel entitled to be inconclusive.
That’s fine in a research lab.
A pilot funded to prove a business case is a capital decision with training wheels. It deserves the same discipline as the rest of the portfolio.
So, what does ownership actually look like here?
Four things—and none of them are technical.
A name.
One executive accountable for the outcome. A committee is not a name. A steering group is not a name. The vendor’s customer success manager is definitely not a name.
A number agreed before the spend.
What are we trying to move, from what baseline, and how will we know?
If the honest answer is, “We’ll see what emerges”, that’s a signal you’re funding curiosity.
Curiosity is a legitimate thing to fund, as long as everybody knows that’s what it is and nobody expects it to appear in the P&L.
A date.
When does someone decide whether to scale it, kill it or redesign it?
Pilots without a decision date don’t end. They fade.
Fading pilots teach the entire organisation that AI initiatives are allowed to go nowhere.
And authority over the workflow.
This is the one that gets missed.
If the accountable owner can’t change how the work is actually done, the tool sits on top of the old process and the return never materialises.
Most of what MIT found looks to me exactly like this: technology dropped into workflows without anybody holding the mandate to redesign them.
None of that requires the board to understand transformer architectures.
It requires the board to do what boards already know how to do: hold management to a defined outcome.
The technology is new. The governance is not.
There’s a harder edge to this too.
When accountability sits with a named leader, failure becomes useful.
A pilot that misses a defined number teaches you something specific about your process, your data or your assumptions.
A pilot that fades with no owner and no number teaches you nothing—and it costs the same.
Over a few years, that difference compounds into the gap between organisations that have learned their way into AI and organisations that have run out of patience with it.
So, here’s what I’d do this week.
Get a list of every live AI initiative in your organisation, including the ones that started as a licence purchase.
For each one, ask three questions:
Whose name is on the outcome?
What number are we trying to move, and from what baseline?
When do we decide what happens next?
If those answers come back quickly, you’re in better shape than most.
If they don’t, you’ve found the reason the pilot won’t show a return months before the review meeting does.
That’s entirely within your power to fix.
That’s all for this week. Thank you for listening or watching, and for being the kind of authentic leader people choose to follow.
I’ll see you next week.
