TL;DR
- AI pilots are usually judged as small versions of production. Analysis suggests they are a different kind of environment, one that runs on support the plan never counts.
- That support, called the pilot subsidy here, includes chosen participants, borrowed attention, a builder who absorbs exceptions, and a forgiving scope. None of it scales by default.
- When a pilot "works" and the rollout disappoints, the usual explanation is the model or user resistance. The subsidy being withdrawn is a better fit for the pattern.
- Organizations that move past the plateau tend to treat the pilot as a measurement of what support was needed, not as proof that support was unnecessary.
The Puzzle: A Success That Does Not Travel
A common sequence in AI adoption goes like this. A small team runs a pilot on a bounded task, such as drafting responses, triaging requests, or summarizing records. Results are good. Participants are enthusiastic. A summary goes upward, and leadership approves a broader rollout.
Six months later the rollout is quiet. Usage is thin or uneven. The same task that went well in the pilot now produces complaints about quality, extra checking, and unclear ownership. The organization concludes that the technology is "not ready," or that staff are "not engaged," and the program drifts into the pattern described in earlier WBA work on why AI working groups produce discussion instead of capability.
The usual diagnoses treat the pilot and the rollout as the same experiment run at two sizes. The argument here is that they are two different experiments. The first ran with a subsidy. The second ran without one.
What the Subsidy Is Made Of
A subsidy, in the economic sense, is a cost someone else pays so that an activity looks viable. Pilots receive at least four of them, and rarely are any recorded.
Selected participants. Pilot members are typically volunteers or people nominated for being curious and capable. They differ from the median employee in exactly the traits that predict success with a new tool: willingness to experiment, tolerance for rough output, and the judgment to know when a result is wrong. A rollout inherits the whole distribution, including people for whom the same output is a risk, not a convenience.
Borrowed attention. During a pilot, someone is watching. The sponsor asks how it is going. The builder checks outputs. Problems get fixed within days because the problem-solver is in the room. This attention is rarely budgeted as pilot cost, since the people providing it are salaried and the hours vanish into other line items. In production, attention returns to its normal level, which is close to none.
An absorbing builder. The person who configured the pilot knows its edges. They quietly route around cases the system handles badly, rewrite prompts when results drift, and handle the odd request by hand. This is a close relative of the dynamic examined in the silent absorber: the exceptions still exist, but a person is absorbing them so the process appears clean. A pilot often has its own absorber, and the pilot's success measures what that person can hold, not what the system can hold.
A forgiving scope. Pilots are scoped to the part of the work that is easiest to demonstrate. Sensitive cases, messy inputs, and edge conditions are excluded, sometimes deliberately and sometimes because the pilot population simply never encountered them. A rollout meets the full range of inputs on its first week.
Why the Subsidy Stays Invisible
If these supports are so consequential, why do they go uncounted? Three structural reasons recur.
First, the pilot's success criteria are set by the people who want it to succeed. A pilot is usually proposed by an advocate, who then defines what a good result looks like. Satisfaction among participants and anecdotes of time saved are easy to collect and flattering. Neither separates the effect of the tool from the effect of the attention around it.
Second, subsidies are experienced as normal conditions by those receiving them. The pilot team does not feel supported, because quick answers and engaged sponsors feel like ordinary good management. The conditions only become visible by their absence.
Third, there is usually no comparison point. As examined in the missing baseline, programs that lack a before-measurement cannot say how much of any improvement came from the tool. Without a baseline, the subsidy cannot be subtracted, because nobody knows what the unsubsidized starting point was.
The result is a decision made on evidence that is real but misattributed. The pilot did produce good outcomes. The error lies in assigning those outcomes to the tool alone and then forecasting them onto a population that will not receive the same support.
A Diagnostic: Three Questions About Any Pilot
A framework-level reading suggests that a pilot's result can be adjusted by asking what would have to be true for it to repeat. Three questions capture most of the exposure.
| Question | What a weak answer reveals |
|---|---|
| Who handled the cases the system got wrong, and how long did that take? | An absorber is carrying hidden cost |
| Would the median person in the target group reach the same result without help? | The result depends on participant selection |
| What happened in the weeks when nobody was checking in? | Attention, not capability, produced the gains |
None of these questions requires new tooling. They require the pilot to record its own support, which is a discipline problem more than a technical one. Organizations that skip the record are not hiding anything; they simply never thought of attention as a resource with a cost.
A fourth consideration sits behind the table: whether the pilot included any case designed to fail. A pilot that never tried a hard input has not established where the boundary is, and a boundary nobody has located is a boundary the rollout will find on its own.
What Separates the Organizations That Move Past It
Within the pattern described here, organizations that get beyond the plateau tend to behave differently at one specific moment: when the pilot ends.
They treat the pilot as a measurement of required support. The question changes from "did it work?" to "what did it take to make it work?" The answer becomes a requirements list. If a pilot needed a builder watching outputs, then a rollout needs a named owner doing the same job, with time budgeted for it. If it needed capable participants, then the rollout needs either training that closes the gap or a narrower first population. If it excluded hard cases, those cases get tested before scaling, not after.
This reframes the scaling decision. The subsidy is not a flaw in the pilot. It is information about the real cost of the capability. A pilot that required a great deal of invisible support is not a failure; it is an accurate preview, provided someone reads it that way.
It also explains an apparent paradox in the three-stage maturity pattern. Many organizations reach a stage where individual use is thriving yet nothing compounds. Pilots keep succeeding, and each success restarts the cycle of enthusiasm without producing a durable operating capability. Each pilot is subsidized afresh, and none leaves behind the support structure that would make the next one unnecessary to subsidize. Maturity, in this reading, is the point at which the subsidy becomes permanent infrastructure instead of a temporary favor.
Limits of the Argument
The pilot subsidy is one explanation for stalled rollouts, not the only one. Some tools do underperform at scale for plain technical reasons, and some rollouts fail because the underlying process was broken before any AI touched it. The claim here is narrower: when a pilot succeeds and a rollout of the same task does not, the withdrawal of unrecorded support should be examined before the technology or the workforce is blamed.
The pattern is also difficult to prove from the outside. It rests on the structure of how pilots are staffed and measured, and each organization's mix of subsidies will differ. What generalizes is the question, not the answer: what was holding this result up, and who was paying for it?
Research questions on this pattern are welcome through Inquiries.