Your AI pilot didn't fail. Your process did.

Two weeks ago, the first Good Option was about writing a receipt for the decisions that matter. This one takes that same idea to where decisions often go unmade: AI pilots. Back in May, I wrote that most stalled pilots aren’t killed by the technology. This is the next step, a four-line map to fill in before your next pilot starts.

AI / Your AI pilot didn’t fail. Your process did.

Most AI pilots don’t end. They fade. The demo goes well, everyone agrees it was impressive, and the item slides to next month’s agenda. Then the month after that.

In May I wrote that the postmortem usually blames the technology, when the real cause was decisions nobody made before the pilot started: what “working” means, what happens if it works, and what would kill it. I still think that’s the diagnosis. But there’s a second problem underneath it, and it shows up even when a team makes those three calls.

The pilot was a capability, not a task

Pilots are usually scoped as a capability: “AI for customer support,” “AI for research,” “AI for the PMO.” A capability isn’t something one person can review or own. It’s a dozen tasks, each with different inputs, different risks, and different people who care. So the pilot gets evaluated as one thing. A few tasks worked well. Others didn’t. The average reads as “promising, but not ready,” and that’s how pilots end up in purgatory.

The AI Workflow Map

The fix is to make the unit of the pilot smaller: one workflow, written down before anyone builds. I call it an AI Workflow Map. Four lines:

• One task. A specific piece of work someone does today, described the way they’d describe it. “Draft first replies for billing tickets,” not “improve support.”

• Its inputs. The information the task needs, where it lives, and whether AI can reach it inside your approved tools and data boundaries. A lot of “the data wasn’t ready” really means nobody wrote this line.

• The review point. Where a person checks the output before it counts, and what they look for. Decide it before the demo, not after the first mistake.

• An owner. One named person who decides whether the workflow continues, changes, or stops. Not a committee. Not “the AI team.”

Then add the three decisions from May at the bottom: what working means for this task, what happens the Monday after it works, and what would make you stop.

How to tell the difference

Here’s the test. Put a stalled pilot next to the map. If the output was good on a specific task, and nobody could say who reviews it or who decides what happens next, the process failed. If the task, inputs, review point, and owner were all clear and the output still wasn’t good enough, the tool may have failed. That’s a real result, and a useful one. It’s also a decision you can write down and revisit when the tools change.

Why so small

A map this narrow can feel unambitious. It isn’t. One workflow tells you whether the inputs are good enough, how long review actually takes, and whether anyone uses the output. A capability-sized pilot hides all three. And when one workflow works, the next one has something real to grow from (see One Good Option below).

I see the same thing on my own side projects. Network Investigator, an open-source Chrome DevTools tab for searching and filtering network traffic, is a long way from an enterprise AI program, but it got useful one small job at a time.

Try it this week

Pick one stalled or upcoming pilot. Fill in the map for the single task where it’s most likely to work. If you can’t name the review point or the owner, you’ve found why it stalled. If you can, run that one workflow for a few weeks before you add a second.

Copy this: the AI Workflow Map

Paste it into a doc, a ticket, or your wiki.

Product / AI makes your system louder

(via Google Cloud DORA, State of AI-assisted Software Development 2025)

DORA’s 2025 report says AI’s primary role is as an amplifier, “magnifying an organization’s existing strengths and weaknesses,” and that the greatest returns come “not from the tools themselves, but from a strategic focus on the underlying organizational system.” The research is about software delivery, but the point carries over to product work. My take: if decisions are slow or ownership is fuzzy before AI arrives, a pilot will make that easier to see, not fix it. Map the workflow before you buy the next tool.

Decisions / Judge the setup before you see the result

(via Annie Duke, interviewed by Stuart Firestein, Nautilus, 2017; a classic)

Edition 1 covered why a good decision can have a bad outcome. Duke adds a name and a practice. The name is “resulting”: “creating too tight a relationship between the quality of the outcome and the quality of the decision,” which is especially risky with a small sample. A single pilot is a sample of one. The practice: when you ask others to judge a decision, leave the outcome out. My take: at your next pilot review, have people rate the setup (task, inputs, review point, owner) before anyone shows the results.

From ProductCamp Conversations

Am I Learning? Not “Am I Doing the Most?” (with Jackie Flake), Oct 17

Takeaway: With AI, the finish line keeps moving, so trying to learn everything is a race nobody wins. Jackie measures herself by whether she’s consistently learning, and her own AI example was small: a family Amazon-spend dashboard she built in about 30 minutes to solve one specific problem.

“Am I learning and am I consistently learning so that not that I can do the most, but that I can maintain that credibility for who I say I’m going to be and who I show up to be in my work.” — Jackie Flake

Listen on YouTube

The Day of the Non-Technical PM May Be Over (with Jason Brett), Oct 23

Takeaway: Jason, who started ProductCamp Atlanta in 2009, says this is a belief he’s abandoned. PMs don’t need to ship code, but AI has made it practical to understand what’s under the hood, and smaller teams make that range more valuable. He’s also adamant that PMs should be building side projects of their own.

“I think they ought to be in the code base and I think they ought to be using AI-powered tools to understand what’s going on under the hood and the implications of what they’re asking for at minimum.” — Jason Brett

Listen on YouTube

ProductCamp Conversations is hosted by Allison Herbert and me. Listen on Apple, Spotify, YouTube, or your favorite podcast app.

One Good Option: Gall’s Law

“A complex system that works is invariably found to have evolved from a simple system that worked.” John Gall wrote that in Systemantics (1975), and added that a complex system designed from scratch never works and can’t be patched up to make it work. (Gall, a pediatrician, retired to Walker, Minnesota.) Use it this week: if your AI pilot is stuck, don’t add scope. Shrink it to one workflow that works, then grow it. (source)

Your turn

Which workflow would you map first? Hit reply and tell me the task.

Which of these six is costing your team the most right now?

• PMs coordinate more than they learn from customers

• Decisions get reopened without new evidence

• Teams work from different practices and assumptions

• We ship, but can’t tell if outcomes improve

• AI experiments never become team practice

• The reasoning leaves when people do

P.S. If this sounds like your team, I do a 25-minute conversation about what’s getting in the way. If I’m not the right fit, I’ll say so. Book a time.

Thanks for reading Good Option. I’m Dave Mathias. I help product teams make better decisions, work with less friction, and put AI to practical use through Beyond the Data. Feel free to connect with me on LinkedIn and tell me what decision you’re wrestling with; I read every one and will reply.

Read past editions at gobeyondthedata.com/newsletter · Forward this to a product leader who’d find it useful.