The pilot worked.
Everyone in the steering meeting agreed it worked. The demo hit its numbers. Someone took a screenshot of the dashboard for a slide deck that got shown to the board.
That was fourteen months ago.
The pilot is still a pilot. Nobody killed it. Nobody greenlit it either. It just — stayed.
The four stages of a pilot that never ships
Stage one: the win everyone can point to
Every stuck pilot starts as a real win. The model worked on the sample it was given. The demo went well. For a few weeks, it’s the best story the data or AI team has to tell — proof that the technology, and the team, can deliver.
Stage two: the question nobody wants to own
Then comes the question that was never assigned to anyone: who decides this goes into production? In a regulated environment — a health plan, an insurer, anywhere audit logging and data residency aren’t optional — that question gets bigger fast. It’s not “does the model work,” it’s “can this survive a compliance review, and who signs off that it can.”
Nobody built that decision into the plan, because the pilot was scoped to prove the model, not to answer that question. So the question sits.
Stage three: the quiet reclassification
Without anyone deciding, the pilot gets relabeled. It stops being “the thing we’re deciding on” and becomes “the ongoing AI initiative” — a line on a roadmap slide that looks like progress. Nobody has to explain why it hasn’t shipped, because on paper, it’s still in motion.
Stage four: the permanent maybe
A year in, the pilot is a fixture. It gets mentioned in quarterly updates. New stakeholders hear about it and assume it’s close. It is not close. It was never given a path to close — just a demo that succeeded once and was never asked to succeed again under real conditions.
Why this happens more in compliance-heavy environments
Health plans and insurers see this pattern constantly, and not because the teams are slower or less capable. It’s because the pilot and the production system are answering two different questions, and only one of them was ever on the plan.
A pilot proves the model works on a sample of claims or member data. Production has to prove the system works under audit — access controls, human-in-the-loop review, data residency, a defensible answer to “what happens when this is wrong.” If the pilot’s architecture was never built with that review in mind, there’s no small next step from “the pilot worked” to “compliance approved it.” There’s a redesign hiding behind a demo that looked finished
(see: The Security Conversation Your AI Team and IT Team Haven’t Had).
That’s a different failure than the one where a pilot gets approved and then struggles in production because of inherited technical debt. This one never gets to that fight — no one ever decided to have it.
The three-question test for whether your pilot is stuck or just early
- Has anyone with the authority to kill or ship this pilot actually reviewed it in the last quarter — not heard about it, reviewed it?
- If the pilot succeeded again tomorrow on real production data, is there a defined next step, or would it just get praised again?
- Could this pilot pass a security or compliance review today, or was it built on the assumption that review would happen “later”?
A no to any one of these usually means the pilot isn’t early. It’s stuck, and it will stay stuck until someone changes the conditions that put it there.
What actually gets a pilot out of purgatory
Momentum doesn’t do this on its own — a pilot that’s been praised for a year has already proven that praise isn’t the mechanism. What moves it is naming an owner with the authority to make the ship-or-kill call, and giving that person a real answer to the compliance question instead of a demo that assumes the question away.
That’s usually a smaller fix than it sounds like from the outside. It’s rarely a full rebuild. More often it’s retrofitting the handful of things a compliance review actually checks — access logging, a human-in-the-loop step, a data residency answer — onto an architecture that was otherwise sound
(see: Broken Pipelines or Broken Ownership?).
This is a different problem than a team running too many pilots at once and diluting focus across all of them
(see: Why Running Too Many AI Pilots Is Making You Slower), and it’s different from an initiative that’s actively in development but sliding past its deadlines
(see: The False Security of “We’re Almost There”). A pilot in purgatory isn’t behind schedule. It has no schedule — because no one ever decided it needed one.
If this is your pilot
If a pilot in your organization succeeded once, got praised once, and hasn’t been reviewed with a real ship-or-kill decision since — it’s not early-stage. It’s stalled, and the stall is structural, not technical.
A focused Data & AI Delivery Efficiency Audit looks at exactly this gap: what the pilot actually proved, what a production decision would require that the pilot never addressed, and what the smallest real path to a yes-or-no answer looks like — instead of another quarter of quiet reclassification
(see: Your AI Team Isn’t Too Small — It’s Doing the Wrong Work).
