Do 95% of AI Pilots Fail? Not in Customer Service.
28 Aug 2025
4.5 min read
A recent MIT report sparked headlines with a daunting claim: 95% of AI pilots are generating “zero return.” According to the lead author of the report, these findings highlight the challenges organizations face in realizing value from AI initiatives. Investors got nervous, executives second-guessed projects, and skeptics felt vindicated.
But as with any bold statistic, the devil is in the details.
What the report actually shows is more nuanced – and if you’re in customer service, it’s even optimistic. In fact, when you dig deeper, customer service is one of the areas where AI is showing the most consistent value. Let’s unpack why the “95% failure” narrative doesn’t tell the full story.
What Does "Zero Return" Really Mean?
The headline figure – that 95% of pilots are stuck – is technically correct. But what it actually refers to is that only 5% of AI pilots are generating multi-million-dollar returns with a direct impact on the P&L.
That's a high bar for success. And it's a bar that excludes productivity gains, automation of back-office tasks, or incremental efficiencies – all of which are very real benefits for most AI deployments, especially in customer support.
Take this quote from the MIT report:
"If I buy a tool to help my team work faster, how do I quantify that impact? I could argue it helps our scientists get their tools faster, but that's several degrees removed from bottom-line impact."
So, if your AI project isn't producing headline revenue gains, it's counted as a failure – even if it's quietly saving your team hundreds of hours.
Why AI in Customer Service Beats the Odds
The 2025 State of AI in Business report highlights that customer service is one of the few areas seeing consistent deployment and measurable returns. That’s because:
Customer support has clear, repeatable workflows that are ideal for automation.
AI in this space can be tied to quantifiable metrics like reduced average handle time, CSAT, and resolution rate.
Unlike ambiguous “AI-powered” efforts in strategy or sales, customer service bots can be measured and optimized in real-time.
Strong model performance in these customer service use cases leads directly to measurable returns.
In other words: customer service is the perfect use case for narrow, high-ROI AI.
What the Report Actually Warns Against
Digging deeper into the MIT analysis, failures stem less from technology limitations and more from organizational and strategic missteps. Poor integration of AI tools within existing systems can result in bad decisions and suboptimal outcomes, further underscoring the need for a cohesive strategy.
1. Unrealistic ROI Expectations
Expecting multimillion-dollar savings from early pilots – especially in customer service – is often a recipe for disappointment. Cost savings from AI scale over time and may be more subtle, such as:
Delaying or avoiding additional headcount
Reducing outsourced BPO spend
Improving internal efficiency
Unless you're operating at massive scale, you're unlikely to see savings that hit the P&L in the first quarter.
2. Investment Bias
The report also highlights “investment bias” – where companies pour resources into high-visibility, top-line initiatives (like AI in sales) rather than high-ROI, back-office functions (like customer service), which are ripe for automation. AI budgets are often allocated to sales and marketing pilots, as these projects are easy to justify and offer quick, visible results. However, sales and marketing pilots are frequently chosen for their accessibility and visibility, even though they may not address deeper operational challenges or deliver substantial ROI. Ironically, this means the projects most likely to succeed often get the least attention.
3. Build vs. Buy Failures
One of the most stark findings: internally built AI projects fail at twice the rate of vendor partnerships. In fact, internal builds succeed only about one-third as often as purchased solutions.
External AI vendors had a ~67% success rate. Internal builds? Closer to 33%. Many enterprises attempt to develop their own proprietary AI systems, but without extensive expertise and resources, internal builds succeed far less frequently.
Why? Because building AI requires:
Deep technical expertise
Continuous iteration and retraining
Workflow mapping and integration
A strong product mindset
Experience with specific use cases
We’ve written about this before, but most companies simply aren’t set up to do this well. Working with specialized vendors can improve success rates, especially in complex or highly regulated industries. Buying from specialized partners accelerates time-to-value and drastically lowers risk.
Specialization > Generalization
Another key takeaway from the report: Narrow AI wins.
Organizations trying to build broad, all-purpose AI tools fared far worse than those who focused on a single, high-impact use case, nailed it, then expanded. This is particularly evident with generative AI projects, which often struggle to deliver value without a focused, use-case-driven approach.
This is especially true in customer service. If you’re in ecommerce, a vendor that deeply understands your specific workflows, customer intents, and integrations will always outperform generalist platforms.
Overcoming the Learning Gap
One of the most overlooked reasons why so many AI pilots fail is the “learning gap”—the disconnect between what AI technology can do and how organizations actually use it. The MIT report calls this out as a core issue: it’s not the quality of the AI models or tools like ChatGPT that’s holding companies back, but the ability of businesses to adapt, integrate, and scale these solutions for real, measurable impact.
Bridging this learning gap is a wake up call for business leaders in the AI era. Too often, AI initiatives are treated as side projects or flashy marketing pilots, rather than as core business solutions. The reality is that success rates for AI pilots skyrocket when companies treat AI as a strategic capability, not just a tech experiment. This means aligning AI projects with business objectives, ensuring that every pilot is designed to solve a real business problem—not just to showcase innovation.
A key lesson from the MIT study is that data readiness is non-negotiable. AI can only deliver ROI if the underlying data is accurate, integrated, and available at scale. Organizations that invest in cleaning up fragmented data and building robust infrastructure see far more measurable returns from their AI pilots. This is especially true in enterprise AI, where back office automation and workflow redesign depend on seamless data flows and system integration.
But the learning gap isn’t just about technology or data—it’s about people and culture. Enterprises that succeed with AI are those that foster organizational readiness: they upskill internal teams, empower line managers to lead adoption, and create a culture where employees are encouraged to experiment, learn, and admit uncertainty. This cultural change is what turns pilots into sustainable solutions that deliver ongoing business impact.
To truly overcome the learning gap, companies must integrate AI deeply into their existing systems and workflows. This means moving beyond shadow AI and generic tools, and instead building or buying solutions that fit the unique needs of the business. When AI is woven into the fabric of daily operations—whether in back office functions or customer-facing roles—it stops being a pilot and starts delivering real value.
In the next decade, the genai divide will separate organizations that treat AI as a strategic, integrated capability from those that chase glamorous areas with zero measurable return. The real story, as the MIT state of AI in business research shows, is that the vast majority of pilots fail not because of the technology, but because of a failure to bridge the learning gap.
The takeaway? For businesses ready to wake up and invest in integration, data readiness, and organizational alignment, the AI era offers enormous potential. Overcoming the learning gap isn’t just about avoiding failure—it’s about unlocking the full ROI that enterprise AI can deliver.
Technology Isn't Failing – The Approach Is
Most failures happen when organizations:
Use static, one-size-fits-all tools
Don’t integrate AI into their core systems or workflows
Expect overnight results without change management
Rely on flashy demos that impress in controlled settings but fail to deliver real, lasting value in enterprise environments
AI isn’t plug-and-play. Success requires tight system integration, continuous learning, and workflow-specific customization.
For instance, it’s one thing for a bot to send a tracking link. It’s another to:
Check the shipping status
Identify delays or delivery failures
Provide a next-best action automatically
That’s the difference between generic automation and true AI enablement.
Our Take: The 95% Failure Stat Doesn't Apply to You
Yes, we’re biased – we specialize in AI for customer service. But our real-world results, especially when it comes to ticket deflection, don’t resemble a 95% failure rate. Not even close.
Most ecommerce brands don’t have the scale to “save millions” through automation – but that’s not the point. A 20-30% reduction in agent load or BPO cost can be transformative in a low-margin business.
So, here’s what does drive success in customer service AI — and why considering metrics like First Contact Resolution matters.
Buy, don’t build – Partner with experts who have done this before
Start narrow – Nail one use case before expanding
Integrate deeply – Don’t just plug AI in – connect it to your systems
Focus on ROI, not flash – Skip the shiny demos and prioritize workflow impact
The real lesson from the MIT research is that success comes from moving beyond generative AI pilots and actually transitioning pilots into impactful, measurable business solutions. It's not just about experimenting with AI, but about delivering real business value.
On the other hand, generative AI pilots in other enterprise areas often fail to deliver measurable ROI, largely due to flawed enterprise integration and limited learning capabilities of generative AI tools.
Looking to Succeed Where Others Fail?
If you're an ecommerce brand exploring AI for customer service, let's talk. We'll show you how to make it work – and avoid becoming part of the 95%.
