MIT Project NANDA’s 2025 report "The GenAI Divide: State of AI in Business" found that around 95% of enterprise generative-AI pilots produced no measurable effect on the P&L. The figure gets quoted as evidence that the technology does not work. That is not what it measures.
What it measures is a delivery failure. The pilots largely did what they were built to do; they were built to show capability rather than to change a process, and capability does not appear in a P&L. The 5% that did show an effect were, overwhelmingly, projects wired into a real workflow with a number attached to them from the start.
If the metric is chosen after the pilot runs, the pilot cannot fail — and it therefore cannot succeed either. Everyone declares partial victory, the project quietly ends, and nothing is deployed.
The fix is boring and effective: name one number, measure it before you start, and agree in advance what value counts as success. "Average handling time on this ticket category, currently 11 minutes, target under 7" is a metric. "Improve customer experience" is a way of avoiding one.
We agree the success metric in the audit, before the pilot is scoped. If we cannot find a number both sides will accept, that is a finding in itself, and usually means the process is not the right first project.
Pilots are demonstrated on the clean 80%. Production is mostly the other 20%: the malformed input, the supplier not in the master data, the customer who is also a partner, the case the rules did not anticipate.
A pilot with no exception path looks impressive and cannot be deployed, because the first day in production produces a case it silently mishandles. Designing the failure paths — retry, route to a person, stop the flow — is not polish added at the end. It is the difference between a demo and a system.
An automation is not a deliverable that stays finished. Suppliers change formats, a system releases a new version, volumes shift, a policy changes. Without a named owner and a dashboard that shows drift, the flow degrades quietly and someone eventually turns it off.
This does not require a team. It requires one person whose job description includes it and a monitoring view that makes degradation visible before a customer finds it.
The most expensive sequencing mistake is proving the model first and connecting the systems afterwards. Integration is where the schedule risk actually lives: credentials, sandboxes, rate limits, a system with no API, a vendor who takes three weeks to answer.
Doing it in the other order — establish that the data can move, then prove the model on real data in place — costs nothing extra and removes the failure mode where a successful pilot cannot be deployed for reasons discovered in month three.
A pilot that runs on exported spreadsheets has not been tested against the thing most likely to kill it.
One process, end to end, on real data, in the real systems, with a named metric measured before and after, an exception path for every step, and an owner from day one. Narrower than the usual pilot in ambition and far wider in what it proves.
That is the whole argument for scoping tightly: a narrow thing in production beats a broad thing in a slide deck, and it is also the only version that produces the evidence needed to fund the next one.