The AI ROI Gap: Why 95% of Enterprise AI Pilots Fail to Pay Off

McKinsey's 2026 State of AI survey of 1,719 respondents found that 37% attribute at least some EBIT impact to AI, and just 6% attribute 5% or more of their organization's EBIT to it and call the impact significant (McKinsey, reported by The Register, August 2026). That gap between AI activity and AI return is the defining problem of enterprise AI. It's also the question your board will eventually ask: what are we actually getting for this?

Many AI programs look the same from the inside. There's an enterprise licence, an encouragement to experiment, a few pilots, and no baseline metric. Activity is high. Return is hard to find.

This guide explains why the AI ROI gap exists, what's changed in the latest research, what AI really costs, and what the winners do differently. It's written for the CFO or steering-committee member who owns, or co-owns, the AI budget. Each section links to a deeper Support Article in this series.

TL;DR

MIT's 2025 research found about 95% of generative AI pilots showed little or no measurable P&L impact, and McKinsey's 2026 survey still puts AI high performers at just 6%. Newer BCG data shows nearly half of companies now generating some value. The difference isn't the model. It's strategy, honest cost accounting, workflow redesign and investment in people.

What Is the AI ROI Gap, and Is the 95% Number Still Real in 2026?

MIT's NANDA initiative found that about 95% of enterprise generative AI pilots delivered little or no measurable P&L impact, while about 5% achieved rapid revenue acceleration (MIT NANDA, reported by Fortune, August 2025). That's the AI ROI gap: the distance between what organizations spend on AI and what shows up in earnings.

The MIT figure became the most-quoted statistic in enterprise AI. It's worth being precise about what it measured. The study drew on 150 leader interviews, a survey of 350 employees and an analysis of 300 public AI deployments. Its headline figure is about pilots, so treat it as a 2025 snapshot rather than a verdict on every AI programme.

What does the newer research say? The gap is real, but it's starting to narrow for organizations that run AI with discipline.

• McKinsey (2026): 37% of respondents attribute at least some EBIT impact to AI, about the same as in 2025. High performers, who attribute 5% or more of EBIT to AI and call the impact significant, are flat at 6% (McKinsey, reported by The Register, August 2026).

• BCG (2025): of 1,250 senior executives surveyed, only 5% of companies were "future-built" for AI, and 60% reported little or no value from their AI investment (BCG, September 2025).

• BCG (2026): a year later, a survey of 1,330 CxOs and senior leaders found 7.5% of companies future-built and a further 41% scaling AI and outperforming, so nearly half now generate value (BCG Applied AI Index 2026, September 2026).

• Gartner: more than 40% of agentic AI projects will be cancelled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls (Gartner, June 2025).

The headline has moved from "AI doesn't pay" to "AI pays for a disciplined few." Read the 2026 surveys together and the picture is consistent. Plenty of organizations now see some return, but the share capturing significant financial value is still in single digits. The question for a CFO is no longer whether AI can pay. It's whether your programme is built like the ones that do.

Gartner adds a warning about the supply side. It warns of "agent washing", the rebranding of existing products as agentic, and estimates only about 130 of the thousands of vendors marketing "agentic" products offer genuine agentic capability. A finance leader evaluating proposals should assume many pitches overstate what's being sold.

Why Does Giving Everyone AI Access Fail?

MIT traced the failure of most pilots to a "learning gap" for both tools and organizations, not to the quality of the models (MIT NANDA, reported by Fortune, August 2025). Access to a capable model is necessary, but on its own it produces activity rather than return.

There's a structural reason for this. General-purpose assistants are flexible, which is exactly why individuals like them. But out of the box they don't know a company's workflows, data or controls. An analyst using a chatbot has to re-explain the context every time. That's a personal productivity aid, not an operating capability.

It helps to picture AI value as a hierarchy:

1. Tool access. Necessary, nearly worthless on its own.

2. Use-case selection. Picking the processes where AI can move a metric.

3. Workflow redesign. Rebuilding the process end to end, not just a task inside it.

4. Organizational change. Training, incentives, review points, adoption.

5. Executive ownership. A named leader accountable for a measured outcome.

Many organizations bought layer one and assumed the other four would follow. The leaders don't work that way. BCG's 2026 index found that leading companies apply AI in high-value areas by reshaping workflows end to end. And at BCG's future-built companies, 88% of managers role-model AI use, against 25% at laggards (BCG, February 2026).

In conversations with finance leaders through the SMU CFO Programme and our own advisory work, one pattern comes up again and again. The organization has an enterprise AI licence. Usage dashboards look healthy. But nobody can name the process that changed, the baseline it was measured against, or the person accountable for the result. That isn't an AI strategy. It's a subscription. The fix starts with choosing where AI should pay off, which is the subject of The Four Gates of AI ROI, our framework for deciding which AI initiatives deserve funding.

The licensing model can make the problem worse. A seat for everyone turns AI from a variable cost tied to output into a fixed cost tied to headcount. We work through the numbers later in this guide.

What Does Enterprise AI Really Cost?

Gartner expects inference, the cost of running a model each time it's called, to account for at least 70% of a model's lifetime cost, and predicts that at least 50% of generative AI projects will overrun their budgets through 2028 (Gartner, reported by Campus Technology, June 2026). The per-token price on a rate card is real, small and a fraction of what you'll actually pay.

The surprise is widespread. In an IDC survey of 318 large organizations, sponsored by DataRobot, 96% of those deploying generative AI said costs came in higher or much higher than expected (IDC InfoBrief, sponsored by DataRobot, December 2025). And McKinsey's 2026 survey found 20% of respondents say AI-related operating costs have already constrained their use of the technology.

Why is the gap so wide? Three mechanics drive it.

Output costs more than input. Gartner puts the output premium at three to five times the input price across providers, and agentic tasks use 5 to 30 times the tokens of a standard chatbot exchange (Gartner, summarized by Kore.ai, 2026).

Context is billed on every call. System prompts, instructions, conversation history and retrieved documents are all input tokens, sent and paid for on every request, even though the user never reads them (Yotta Labs, May 2026).

Tokenizers differ. For languages with complex writing systems, such as Tamil, RWS measured up to a 450% difference in token usage between the least and most efficient tokenizers. In its worked example, a financial-services firm handling 100,000 customer inquiries a day pays about $100 a day on an efficient model and about $450 a day on one that needs 4.5 times the tokens (RWS, December 2025). Over a year, that's roughly $36,500 against $164,250 (AIforCFO arithmetic).

Then come the layers around the token bill: retrieval infrastructure, evaluation and human review, monitoring, and the engineers who keep the system running. In AIforCFO's worked example for one finance workflow, those layers make the full run cost about 3x the API bill, and 2x to 4x across a range of assumptions (AIforCFO analysis, 2026).

Cheaper AI is making AI bills bigger, not smaller. Epoch AI found the price of reaching a fixed level of model performance has fallen between 9x and 900x a year, depending on the benchmark (Epoch AI, March 2025). Yet Gartner predicts inference costs per agentic workflow will rise more than fivefold through 2028 (Gartner, reported by Techstrong.ai, August 2026). That's the Jevons paradox: cheaper tokens invite longer contexts, more agents and more usage.

Before signing an AI business case, a CFO should require four things:

• A consumption forecast, not a seat count. Tokens per workflow at production volume, including context and reasoning, plus the run-cost layers around the API bill.

• A routing plan. Send routine tasks to small models and reserve premium models for hard ones. Gartner, as cited by Kore.ai, reports contextual routing savings of 37-46%.

• A FinOps control plane. Real-time token tracking, per-team budgets and alerts.

• A kill criterion. Usage and value thresholds, agreed in advance, below which the deployment is switched off.

For cost models and a business-case template you can adapt, see ROI of AI in Finance: Benchmarks and Business Case Guide.

Where Does AI ROI Actually Concentrate?

MIT's GenAI Divide research found more than half of generative AI budgets go to sales and marketing tools, while the biggest ROI showed up in back-office automation (MIT NANDA, reported by Fortune, August 2025). The money and the value are pointed in different directions.

Why does this happen? Visible use cases get funded first. A sales copilot demos well in a leadership meeting. An invoice-exception workflow doesn't. Buying a licence is fast and feels like progress. Process analysis is slow, unglamorous and politically awkward, so it gets skipped.

Finance is a good example of how hard the payoff is to prove. In BCG's March 2025 survey of more than 280 finance executives, the median reported ROI from AI and generative AI was 10%, below the 20% many were targeting, and only 45% could quantify ROI at all (BCG, June 2025).

BCG also gives finance teams a useful filter: use generative AI where the work is about language, not maths. Drafting, synthesis, classification and summarization suit it. Deterministic tasks such as reconciling accounts are better handled by rule-based automation or traditional AI, with auditable outputs.

A simple two-axis score finds the targets:

Axis 1: Value potential	Axis 2: AI suitability
Volume and frequency of the task	Language-based vs. calculation-based work
Cost of the current approach	Tolerance for probabilistic output
Cycle-time sensitivity	Quality of data available to ground the model
Strategic visibility of the outcome	Degree of workflow integration required

High-volume, high-cost, language-heavy processes with clean data are where you start. Choosing one such process and rebuilding it end to end beats spreading licences across the whole organization.

How Does Shadow AI Erode AI ROI?

LayerX's browser telemetry found 45% of enterprise employees use generative AI tools, 77% of those users paste data into them, and 22% of those pastes include personal or payment-card information. 82% of paste events come from unmanaged personal accounts (LayerX, reported by The Register, October 2025).

That last number is the one to remember. Personal accounts sit entirely outside corporate visibility and controls. An enterprise agreement that stops a provider training on your data is necessary but not sufficient. It doesn't stop someone pasting a client file into a personal account on their phone.

Why is this an ROI problem and not only a security problem? Because broad, uncontrolled access creates cost without control. A data incident adds remediation, legal and trust costs that never appeared in the business case. And when leadership responds by banning tools outright, it writes off the productivity case those tools were bought for.

The better answer is a sanctioned, well-governed alternative that's easier to use than the personal account, plus clear rules on what data may go where.

Why Do People, Not Models, Decide AI ROI?

BCG's 10-20-70 rule holds that about 10% of AI value comes from algorithms, 20% from the technology needed to implement them, and 70% from rethinking the people component (BCG, February 2026). In one line, it explains why buying a licence fails. The licence sits inside the 10%.

The budgeting implication is direct and often ignored. If 70% of value comes from people and process change, a credible AI budget puts a comparable share of effort into workflow redesign, training and adoption. Many organizations invert this. They spend the bulk on technology and a token amount on people. Then they wonder why adoption is shallow.

The leaders invest in people at a different scale. BCG found future-built companies plan to upskill more than 50% of their employees on AI, against 20% for laggards, and are four times more likely to run structured AI-learning programmes with protected learning time.

What does the 70% actually contain?

• Strategic alignment. Leadership making sure AI serves enterprise priorities, not a walled-off innovation lab.

• Behavioural change. Role-based learning that changes daily work, not a one-off training session.

• Workflow redesign. The same point that surfaces in every section, because it's the crux.

• Incentives, measurement and review. Rewards that reinforce adoption, metrics that track the shift, and review points designed in from day one. In finance, unreviewed AI output isn't a productivity gain. It's a control failure waiting to be found.

Finance needs a specific combination: domain judgement plus the literacy to frame, supervise and check model output. A professional who can't validate AI output can't safely deploy it.

For a practical sequence that builds team fluency alongside the technology, see AI for CFOs: The 90-Day Roadmap.

Should You Build, Buy, or Go Hybrid, and How Do You Avoid Lock-In?

MIT found that purchased tools from specialized vendors, combined with partnerships, succeed about 67% of the time, while internal builds succeed only a third as often (MIT NANDA, reported by Fortune, August 2025). The lesson isn't "never build." It's "don't build the undifferentiated parts."

A practical decision rule:

If this is true	Default path
The capability isn't your competitive edge	Buy or partner
Proprietary data or process is the differentiator	Build the thin layer that's genuinely yours
Sensitive data plus a residency or compliance mandate	Private or open-weight deployment
You need frontier capability at variable volume	Public API
You want control and predictable cost at scale	Hybrid
Any of the above	Keep governance, abstraction and evaluation in-house

Private and open-weight models are a real option, but they aren't free. Self-hosting carries infrastructure, security and talent costs, and it only pays at high, steady volume or under a hard compliance requirement.

Lock-in is the other strategic risk to price in. Menlo Ventures' mid-2025 data showed Anthropic with 32% of enterprise LLM usage, OpenAI 25%, Google 20% and Meta's Llama 9%, with 13% of AI workloads on open-source models (Menlo Ventures, July 2025). Menlo also found only 11% of builders switched vendors in the past year, while 66% upgraded within their existing provider. Switching is easy in theory and rare in practice. That's exactly the condition under which a provider gains pricing power.

The defence is architectural. Route requests through a model-agnostic gateway. Keep a second model option live as a cost control and a fallback. Maintain a portable evaluation suite so you can qualify an alternative model quickly. For most enterprises, a hybrid architecture is both the cost optimum and the best lock-in insurance available.

What Does a Worked AI Cost Comparison Show?

In AIforCFO's worked example, blanket seat licensing for 5,000 employees costs about $1.8 million a year, while targeted licensing for the 500 people who use AI heavily costs about $180,000 (AIforCFO analysis, 2026). That's roughly $1.6 million a year spent on seats that create little value, and it's one of the easiest costs in enterprise AI to miss.

Here's the same mid-sized finance-operations workload priced four ways. The numbers are illustrative and rounded, not a vendor quote. The assumptions are stated below so you can challenge them and swap in your own.

Delivery model	Key assumptions	Illustrative annual cost
Blanket seat licensing (all 5,000 staff)	$30 per seat per month, fixed regardless of use	~$1,800,000
Targeted seat licensing (500 active users)	$30 per seat per month, heavy users only	~$180,000
Metered API (500 active users)	~$0.03 per run; 2.5 million runs a year; run cost 3x the API bill	~$225,000
Private / open-weight hosting (mid-tier)	~$25,000 a month of infrastructure plus ~$200,000 a year of operations staff	~$500,000

Assumptions: 5,000 employees; 500 heavy users; about 20 workflow runs per active user per working day; about 250 working days; seat price and hosting costs are illustrative assumptions, not quotes; the metered API line uses the central case from our TCO worked example. Source: AIforCFO analysis, 2026.

What do the numbers say? At this scale, targeted licensing and metered API are the efficient options. Private hosting isn't the cheapest at 500 users. At about $500,000 it earns its place on data sovereignty, compliance, or volumes well above this scenario, not on cost alone.

One caveat keeps the comparison honest. None of these figures include data readiness, reskilling or governance costs. Those are often the largest first-year lines, and they're excluded here to keep the delivery-model comparison clean. Don't mistake this table for a full business case.

What Do the Winners Do Differently, and What Should a CFO Do Next?

BCG's 2026 index found future-built companies deliver 2.3 times the total shareholder return, 2.4 times the revenue growth and 2.8 times the EBITDA growth of AI laggards (BCG Applied AI Index 2026, September 2026). The same research found AI spending rising from about 1.7% of revenue in late 2025 to 3.3%. The difference isn't how much access they bought. It's how they allocate capital.

The winners share a recognizable profile:

• They concentrate. They pick a few high-value workflows rather than scattering effort across dozens of pilots.

• They redesign workflows end to end, BCG's recurring finding.

• They invest in people, upskilling far more of their workforce than laggards do.

• They lead from the top, with managers who use AI themselves.

• They measure and keep what works. Gartner found 45% of organizations with high AI maturity keep AI projects in production for three years or more, against 20% of low-maturity organizations (Gartner, June 2025).

The AI ROI gap is, at bottom, a capital-allocation failure. That makes it the CFO's problem to own, not IT's. Every discipline that closes the gap is already standard practice in the Office of the CFO: a baseline before investment, an honest total cost, concentrated bets, controls and review points, and a kill criterion. These disciplines simply haven't been applied to AI yet, because AI arrived dressed as a technology purchase rather than a capital decision.

A CFO checklist for every AI investment

1. Tie the case to a process and a baseline. Cycle time, error rate or cost per unit, measured before AI. No baseline means no ROI, only anecdote.

2. Price the full run cost from a consumption forecast, not a seat count or a rate card.

3. Enforce language-not-maths discipline. No probabilistic model owns an exact figure without a deterministic check and human review.

4. Treat data exposure as a board-level risk. Give people a sanctioned tool and keep sensitive data out of personal AI accounts.

5. Fund the 70%. Budget for reskilling and workflow redesign alongside tooling.

6. Write the kill criteria before launch. Not after the first overrun or the first incident.

If your current programme is an enterprise licence plus encouragement to experiment, you've bought a lottery ticket, not a strategy. The good news is that the fix doesn't need new technology. It needs finance discipline applied earlier.

This guide is the hub of our AI ROI Gap series. Deep dives on total cost of ownership, shadow AI, build vs. buy, vendor lock-in and AI skills are on the way. Browse all articles in the meantime.

Frequently Asked Questions

Is the MIT 95% AI failure statistic accurate?

It was a 2025 snapshot of pilots, not whole programmes. MIT NANDA found about 95% of generative AI pilots showed little or no measurable P&L impact. Newer data is more mixed: McKinsey's 2026 survey still puts high performers at 6%, while BCG's 2026 index finds nearly half of companies now generate some value from AI.

What is the real total cost of ownership of enterprise AI?

It's well above the token bill. Gartner expects inference to make up at least 70% of a model's lifetime cost, and IDC found 96% of generative AI adopters saw costs above expectations. In AIforCFO's worked example, the full run cost is about 3x the API bill once retrieval, evaluation, tooling and engineering are included.

What is the 10-20-70 rule in AI?

BCG's 10-20-70 rule says about 10% of AI value comes from algorithms, 20% from technology and 70% from people and process change. For budgeting, it means training and workflow redesign deserve serious funding alongside tools. BCG found future-built companies plan to upskill more than 50% of employees, against 20% at laggards.

Should every employee get an enterprise AI licence?

Usually not. In AIforCFO's worked example, licensing all 5,000 staff costs about $1.8 million a year versus about $180,000 for the 500 heavy users. Targeted access, or metered API usage, is typically the efficient choice. Broad unmanaged use also raises shadow-AI exposure: LayerX found 82% of data pastes into AI tools come from personal accounts.

How long does it take for an AI investment to pay back?

It depends on the process, not the tool. Gartner found 45% of high-maturity organizations keep AI projects in production for three years or more, against 20% of low-maturity ones. In finance, BCG's survey put median AI ROI at just 10%. Payback comes from redesigned workflows measured against a pre-AI baseline.

Conclusion: AI Amplifies Discipline, It Doesn't Replace It

The belief that giving everyone access to an AI assistant will produce return is a central reason so many organizations are spending without returning. The latest research shows return is possible, but significant financial value still goes to a small, disciplined group.

Key takeaways:

• The gap is real but moving. High performers are still 6% (McKinsey 2026), while nearly half of companies now see some value (BCG 2026).

• The rate card is a fraction of the cost. Budget for the full run cost.

• Value sits in mapped, redesigned processes, often in the back office rather than where most budgets go.

• Shadow AI turns broad access into uncontrolled cost and risk.

• 70% of value is people and process. Fund it.

AI pointed at a mapped, redesigned, well-governed process with trained people behind it compounds value. Handed to everyone with no strategy, it compounds cost and risk instead. That choice gets made in the C-suite, before a single licence is signed. Start with the process. The AIforCFO Diagnostic applies the Four Gates framework to your own finance team and ends with a 90-day plan.

A note on shelf life. The research and prices here were checked in October 2026, ahead of publication in November 2026. Survey figures are updated every year, and AI vendors regularly change their pricing, bundling and licence terms, so some figures will have moved by the time you read this. The disciplines in this guide should hold.