Bosses see no gain from AI, staff swear it saves time. What the evidence says works
Jamie Watters
Operational resilience and AI delivery practitioner.
Who this is for: you're putting AI to work in an organisation, paying someone else to, or advising people who are. This paper pulls together what the evidence says about each part of the job, from choosing who does it to what happens to the people around it. By the end you'll know which common advice rests on evidence, which rests on a vendor's say-so, and you'll have a six-step method you can run on one process.
Skip it if you want tips for using a chat assistant on your own work. This is about changing how work gets done when other people depend on it.
On the length: it's long, because it covers nine areas. The summary below gives you the findings in two minutes. Each numbered section stands on its own, so read the ones you need. The method you can copy is in section 11.
Summary
In May 2026 the two biggest AI labs both went into the services business. OpenAI launched a company to put its models to work inside other firms, with "more than $4 billion of initial investment". Anthropic announced a services venture with Blackstone, Hellman & Friedman and Goldman Sachs. OpenAI's chief operating officer, Brad Lightcap, was quoted giving the reason: customers "need help going from pilot to production".
That's the problem this paper is about. Getting AI from a trial into the way an organisation actually runs is where the labs themselves say customers need help. I wanted to know what the evidence says about that step, so I put the question through the research process I use for everything. It pulls the primary sources, grades each claim by how good its evidence is, and keeps the authors' own caveats with the claim.
The evidence comes in two kinds, and they rarely meet. There are tightly designed but narrow studies: randomised trials of AI on single tasks, and older research on checks and controls in medicine, aviation and banking. And there are broad but weak sources: surveys and consultancy reports on "AI transformation". This paper uses both, and says which is which.
What it found, in short:
- The usual signs that AI is working can mislead. Staff report time saved that a stopwatch doesn't find. Nine in ten of nearly 6,000 executives see no effect on productivity or jobs (Yotzov, Bloom and colleagues, 2026). Where government auditors looked, at the US tax authority and the Australian Taxation Office, they found no regular monitoring of the AI once it was live.
- The person checking the AI is the weakest safeguard. In a study with a simulated AI, very experienced radiologists were far less accurate when its suggestion was wrong. Training doesn't fix this. Measuring the reviewers is the best-supported answer.
- Nobody has compared the ways of implementing AI. Vendor engineers, consultancies, in-house teams and staff left to it all have their advocates. No study in my research compares them on results.
- The written process is rarely the real one, and the system records that are meant to show the real one have gaps of their own.
- Where gains have been measured properly they're real, uneven and modest. Output per worker rose in several trials. Pay hasn't followed in the best evidence so far, and firms report payback taking years, not quarters.
- The tests that work already exist, in hospitals, airports and banks. Most of what the evidence says about making AI pay comes from those fields, not from AI itself.
Building the AI is the easy part. What the evidence backs is the set of checks around it.
Where I'm coming from: I've worked in financial services since 1987, latterly in business continuity and resilience. From 2023 I was a member of a bank's committee that approved AI and cloud initiatives for development and deployment. Each discipline signed off its own area: information security, infrastructure, risk, legal, compliance and finance. Most of what came to it was about data in the cloud, hosting, resilience and data moving offshore. I didn't design that process. I was a member of it.
1. What "implementing AI" means
The phrase covers three different things, and most headline numbers mix them up.
One person using a general tool, such as a chat assistant, on their own work. This is the best measured, with proper randomised trials and national pay records behind it.
A process redesigned around AI, where some of its steps are now done by a model. This is the job of the engineers the AI firms send into companies, and of consultancies. It has the thinnest direct evidence of the three.
A company saying it "uses AI". This is what surveys count, and the definitions keep moving.
Most headline figures are about the third. The method that matters is about the second. So much of the evidence for the method comes from older fields, carried across. This paper says so each time it does that.
2. How often it works, and what the successful ones have in common
How many firms use AI, and how deeply
The US Census Bureau asks firms whether they used AI in the past two weeks. From December 2025 to May 2026, between 17% and 20% said yes, with 19.8% at 3 May 2026. In finance and insurance it was 33.9%, and among firms with 250 or more staff it was 37%. The Bureau changed its question in November 2025, from use "in producing goods or services" to use "in any business function", so figures from before and after can't be compared. Most statistics websites drop that break.
The use is shallow. A Census working paper from April 2026 found that 18% of firms used AI in at least one business function (32% when weighted by employment). Of those, 57% used it in three functions or fewer, and 66% used it only to help with existing tasks, not to replace them (Bonney and colleagues, CES-WP-26-25). Job cuts linked to AI happened at 2% of firms.
A 2026 survey asked nearly 6,000 executives in the US, the UK, Germany and Australia about AI (Yotzov, Barrero, Bloom, Davis and colleagues, NBER working paper w34836). 69% said their firms used it. Nine in ten reported no effect on employment or productivity over the previous three years. They expect a 1.4% productivity gain over the next three years. A separate survey of about 750 executives found that the gains managers perceive are bigger than the gains measured (Baslandze and colleagues, NBER w34984, March 2026).
What the successful deployments have in common
Very little evidence measures this directly. Three studies come closest.
On one online retail platform, seven uses of generative AI were tested at random. Four raised sales, by up to 16.3%. The rest showed no detectable change. The gains depended on how much the AI added beyond what the platform already did (Fang and colleagues, arXiv 2510.12049, a preprint; one author worked for the platform and two consulted for it). The lesson: which work you choose matters more than how well you build it.
At one firm told to double its software output, AI tools raised the amount of code each developer submitted by roughly one and a half times or more. The gain was about 44% in new code and about 12% in older code, and the 12% wasn't statistically reliable. Quality was checked only through rough measures, how often code was merged and how often it was reverted, and both held steady (He and colleagues, arXiv 2607.01904, July 2026, a preprint).
The consultancy BCG reports (September 2025) that the 5% of firms it calls "future-built" get 62% of their AI projects into use, against 12% for the laggards, and that they redesign whole workflows. That's a correlation from a firm that sells the redesign. My research found no randomised study across firms that tests whether redesigning workflows is what makes the difference.
3. Who does the implementing
AI gets into an organisation in five main ways. Each has advocates. My research found no study that compares them on results.
Every claim that one way beats another is either self-reported or comes from the people selling it.
Engineers sent in by the AI firm
These are called forward deployed engineers: engineers the AI company places inside a customer's business to build with its models. Palantir made the model famous. Anthropic and OpenAI now hire for it.
The job adverts say what the job is. Anthropic's forward deployed engineers build "production applications with Claude models" inside customer systems, with a quarter to half of their time travelling. Anthropic also advertises a separate Technical Deployment Lead, who owns the delivery plan and "value measurement and ROI" and doesn't write production code. OpenAI's forward deployed engineers "own technical delivery... from first prototype to stable production". Palantir's engineers for US government work are "often on site". None of the adverts says how much of the job is coding and how much is working with the customer. These are the firms' own adverts, so they describe the job as the firms want it seen.
Why now? The firms' own statements give the answer. Anthropic's chief financial officer, Krishna Rao, said demand is outpacing "any single delivery model". OpenAI has agreed to buy Tomoro, which would bring "approximately 150" forward deployed engineers and deployment specialists; the deal was announced subject to regulatory approval. Salesforce's latest annual report mentions forward deployed engineers for the first time, there to get its AI agents "operationalized rapidly". The plain reading: the AI firms' products are general and need shaping to each customer, and customers can't do that shaping on their own.
This model is expensive for the vendor to run. Palantir's own annual report warns that its intensive pilot workshops "may result in no or minimal future revenue". What it costs a buyer is harder to pin down: no filing in my research separates the cost of embedded engineers from the rest of a contract.
The claim you'll hear most about this job, that postings are "up 800%", traces to a Financial Times report citing a job-listings site. My research could open neither the report nor the data.
Consultancies
Consultancies bring scale and experience of managing change. The evidence on what they cost is less flattering. Research on NHS hospital trusts found that spending on consultants went with lower efficiency (Kirkpatrick, Sturdy and colleagues, 2019), and that using them was associated with "demand inflation", more demand for consultants (Sturdy and colleagues, 2022). In Australia, Deloitte part-refunded A$97,587 of an A$440,000 government report that contained fabricated references produced with an AI toolchain.
Accenture booked $5.9bn of new generative AI work in its 2025 financial year. It reported the figure for one more quarter, then said it would stop.
The vendor's own services team
My research found no AI-specific figures for this in any company filing. ServiceNow says it "increasingly" relies on partner firms for services.
An in-house team
Whether a central team works better than teams spread across the business is untested. My research found no quantitative comparison. A research project at MIT on the question is still in progress, framed around "incremental productivity gains... but no positive cash flows".
Staff left to get on with it
This gets the widest reach. In Denmark, employer initiatives such as encouraging AI use nearly doubled take-up for a typical worker, from 47% to 83% (Humlum and Vestergaard, April 2025 version; the later revision reports the same pattern without those two figures). Telling workers about the benefits changed what they believed, but not whether they used it.
Use is uneven. Women were about 20 percentage points less likely than men in the same job to use ChatGPT (IZA discussion paper 16992, 2024). And as section 8 shows, time saved this way hasn't shown up in pay.
Microsoft's figure that 78% of AI users "bring their own AI" to work comes from a survey it commissioned: 31,000 knowledge workers in 31 markets, run by Edelman Data & Intelligence in February and March 2024. It's a vendor's online survey, so read it as that.
Building versus buying
An MIT report (the NANDA report, 2025) says external partnerships with vendors "reached deployment ~67% of the time, compared to ~33% for internally built tools". That comes from interviews with 52 organisations, not measured results, and the authors say it "may reflect organizational capabilities rather than implementation approach alone".
4. Finding how the work is really done
Before any AI goes in, someone has to find out how the process actually runs. The evidence says the written version is rarely the real one, and the system records meant to show the real one have gaps too.
What the records show, and what they miss
Process mining builds a picture of a process from the time-stamped records a system keeps. Its founding principles assume those records show what "actually happened" and are complete (the IEEE Process Mining Manifesto, 2011). In practice, a study of five real sets of records found 27 kinds of quality problem in them (Bose, Mans and van der Aalst, 2013).
Real processes are messier than their documents. One hospital study compared the official guideline for COVID-19 care with what actually happened to 187 patients (Benevento and colleagues, a preprint). Across three waves of the pandemic, the guideline scored only 0.66 to 0.69 out of 1 for how well it fitted. The records from a gynaecological cancer department produced 376 different kinds of event and a tangled process map (Mans and colleagues, 2008).
What the records can't see at all, such as work done in email, on paper or in side spreadsheets, is widely assumed but hasn't been studied directly. It's an inference from the process-mining literature and from research on workarounds, which one researcher called "understudied and undertheorized" (Alter, 2014). My research found no study that measures it.
Interviews and watching people work
Asking people what they do, and watching them do it, fill the gaps. The classic study here found that office work is organised in what people actually do, not in "procedural specifications" (Suchman, 1983). But it rested on one episode, two workers and one office, and the author called it "preliminary". The idea that work as imagined differs from work as done is widely used, but its originator said the gap "may or may not be problematic" (Hollnagel, 2017). The popular version treats every gap as a problem.
How well do interviews match what people actually do? The only measured comparison my research found was for physical work: self-reports matched expert observation well for lifting, and poorly for most other tasks (Dale and colleagues, 2010). My research found no such measurement for office work.
How long this should take
The UK government's service manual says "There's no set time period for a discovery, but around 4 to 8 weeks is typical", and that you "should not start building your service in discovery". That's guidance. My research found no study of how long this step should take.
What held up from "re-engineering"
In 1990 Michael Hammer argued that companies should "obliterate" outdated processes rather than automate them. His example was Ford's accounts payable, which cut its headcount by 75%. That core idea, redesign before you automate, has held up. What didn't hold up was re-engineering's slide into cost-cutting: one of its own early champions, Thomas Davenport, wrote in 1995 that cutting costs alone was "not a sensible goal".
The famous statistic didn't hold up either. Hammer and Champy's 1993 book is reported in secondary sources as saying "our unscientific estimate is that as many as 50 per cent to 70 per cent" of re-engineering efforts fail. Hammer was later quoted saying the figure had been "transmogrified and distorted into a normative statement". A review of the sources behind "70% of change programmes fail" found "no valid and reliable empirical evidence" for it (Hughes, 2011).
5. Deciding what AI should do
Once you know the real process, each step needs an owner: a fixed rule, an AI, or a person. Or nobody, because the step should go.
Cut, simplify, rule, AI or person
Asking whether a step can be eliminated, combined, rearranged or simplified goes back to the US government's Training Within Industry programme of the 1940s. Ask those questions first.
Fixed rules follow "if this, then that". A review of 63 studies found they suit work with four features: high volume, a standard and stable process, few exceptions, and digital inputs and outputs (Wewerka and Reichert, 2020). Much of the evidence behind it comes from single-company case studies.
What's different about generative AI
Three things.
Its errors are uneven, and hard to predict. In a trial with 758 consultants, AI raised the quality and speed of work on tasks it was good at. On a task it wasn't good at, consultants using AI were 13.9 to 24.5 percentage points less likely to get the right answer (Dell'Acqua and colleagues, Organization Science, 2026). Yet their answers were rated as more coherent.
The gains go mostly to the least experienced. In customer support, an AI assistant raised resolutions an hour by 15% on average (Brynjolfsson, Li and Raymond, Quarterly Journal of Economics, 2025). Less experienced and lower-skilled agents gained most. The most skilled gained a little speed and lost a little quality. Less experienced software developers also took up AI tools more and gained more (Cui and colleagues, Management Science).
Help can harm the people who are struggling. In a field experiment in Kenya, an AI mentor for small business owners had no detectable effect on average. Those who had been performing poorly did nearly 10% worse; those who had been performing well "may have benefited by over 15%" (Otis and colleagues, Management Science, 2026). The authors hedge the second finding. The popular version doesn't.
Separating doing from deciding
Research on automation splits any task into four parts: gathering information, analysing it, deciding, and acting. Each part can be automated to a different degree (Parasuraman, Sheridan and Wickens, 2000). A review of 18 experiments found that more automation improved routine performance but made things worse when the automation failed (Onnasch and colleagues, 2014).
That's the evidence for separating the doing from the approving: put the strongest checks on deciding and acting, in proportion to what a mistake would cost. Gathering and analysing aren't free of risk either. They can expose confidential data, so they need access controls of their own.
Where a person stays
The law says so in some places. The EU AI Act requires human oversight of high-risk AI, including awareness of "automation bias", the power to override or reverse the system, and a way to stop it. In the US, the Federal Reserve's 2011 guidance on bank models (SR 11-7) asked for "effective challenge": "critical analysis by objective, informed parties who can identify model limitations and assumptions". Its April 2026 replacement, SR 26-2, says in a footnote that "Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance", while other AI models stay in. In the UK, the Prudential Regulation Authority's supervisory statement on model risk covers AI used in models and requires a named senior manager to own model risk, but only for banks approved to use their own models to set capital. The Financial Conduct Authority says its rules on senior managers' accountability cover any use of AI, and describes itself as "technology-agnostic": its rules "do not usually mandate or prohibit specific technologies".
The rule that a person should keep any step where a mistake is expensive or can't be undone is sensible. The nearest research source is Parasuraman and colleagues' test of what the consequences would cost. My research found no study that states it as a rule or tests it.
6. Building it
The common advice is to build AI inside the software people already use, rather than giving them a new tool. The hospital evidence on this is split.
A review looked at 70 trials of software that gives doctors advice (Kawamoto and colleagues, BMJ, 2005). When the advice appeared automatically while the doctor was working, it worked in 75% of trials. When the doctor had to go and look for it, it worked in none, though few trials tested that. The same review found that advice built into the patient record or ordering system worked more often, 73% against 36%.
A larger review, of 162 trials, found the opposite on that second point: advice built into the electronic patient record or ordering system was less likely to show it worked (Roshanov and colleagues, BMJ, 2013). And in that larger review, advice arriving automatically or at the time of care made no significant difference, so the authors dropped it from their final model. Making doctors give a reason when they overrode the advice was strongly linked to it working. And trials run by the people who built the software were more likely to report success, which is a warning about who does the measuring. The review is observational, so these are associations.
So the two reviews disagree on whether delivery at the moment of decision is what makes the difference. Putting the AI's answer where the decision is made is a sensible design choice, supported by some of the evidence, not a settled finding. One point holds across this research. A tool has to be used, and it has to fit the task: research on information systems found that tools improve performance only when they're both used and suited to the work (Goodhue and Thompson, 1995).
Real conditions break models built on clean data. When Google tested its eye-screening AI in 11 clinics in Thailand, the lighting and image quality caused the model to reject images it had been designed to read (Beede and colleagues, 2020, Google's own researchers reporting on their own system).
7. Testing it before you trust it
Running old and new side by side
Some banks already have to do this. The UK's model risk rules expect banks with approved internal models to compare each model's answers with those of a "challenger", a separate model built to check it. They also expect "parallel outcomes analysis": running old and new side by side and comparing the results. They also expect ongoing monitoring of models bought from vendors. Google's engineering practice of "canarying" releases a change to a small part of the system, compares it with the rest, and widens it only if it holds up.
My research found no study that measures the results of running AI alongside an old process. The practice is required in banking and common in software. Its effect on AI deployments is untested.
Testing the people who check
Airport security tests its screeners. EU aviation security rules have required screening systems to project fictional threat images into the bags being screened, with the mix of image types adjustable. Each screener's results are stored for at least 12 months, and the screener is told when they miss one. A 2025 study of nearly 1.2 million projected images and 728 screeners found these scores predicted how screeners did in covert tests, where real prohibited items were smuggled through (Buser and colleagues, Transportation Research Part A). That shows the scores are a valid measure, with a condition: it took about 100 sufficiently hard test images per screener before an individual's score was reliable. Whether projecting the images keeps screeners more alert is a separate claim that nothing I found tests.
Why test at all? Because rare things get missed. In one lab study, people searching for targets missed far more when the targets were rare than when they were common (Wolfe and colleagues, Nature, 2005; the exact figures come via secondary sources). In real breast screening, six radiologists looked for cancers planted in scans (Evans, Birdwell and Wolfe, 2013). When cancers were as rare as in normal practice, they missed 30%. When the same cancers were common, they missed 12%. If an AI is right nearly all the time, its mistakes are rare, and rare mistakes are the ones people miss.
Software testing uses the same idea: deliberately plant faults and see whether the tests catch them. Research has found that catching planted faults goes with catching real ones (Just and colleagues, 2014). My research found no evidence of planted errors being used to test reviewers in audit or financial quality checks. That's a gap, and an opening.
Why "a human checks it" isn't enough
People defer to automated advice, and it doesn't take inexperience. A review of the research concluded that automation bias affects experts and novices alike and "cannot be prevented by training or instructions" (Parasuraman and Manzey, 2010).
Accuracy held when the AI was right and fell when it was wrong, at every level of experience.
In one study, 27 radiologists read mammograms with an AI's suggestion on screen (Dratsch and colleagues, Radiology, 2023). When the AI was right, the very experienced readers were right 82.3% of the time. When it was wrong, they were right 45.5% of the time. The AI was simulated, there was no group reading without it, and only five readers were very experienced. Popular versions drop all three caveats.
Explanations can make it worse. In one study, AI explanations made people more likely to accept the AI's answer, whether it was right or wrong (Bansal and colleagues, 2021).
A study of 41 government policies requiring human oversight of algorithms found that people "defer to automated systems", and that the requirements "provide a false sense of security" (Green, 2022).
What helps
Some things measurably reduce people going along with wrong answers.
- Making people think before they see the AI's answer reduced over-reliance, though people liked it least (Buçinca and colleagues, 2021, 199 participants).
- The AI saying it isn't sure, in the first person, reduced over-reliance without removing it (Kim and colleagues, 2024, 404 participants, registered in advance).
- Requiring a reason to override was strongly linked to advice working, in the 162 hospital trials above.
- Feeling accountable for the outcome went with more checking and fewer errors among pilots, though that study found a correlation, not a cause (Mosier, Skitka and colleagues, 1996).
- Showing information rather than a bare recommendation is one of the things that reduce automation bias, according to a review of 74 clinical studies (Goddard and colleagues, 2012).
Volume works against all of it. At one firm, after AI coding tools arrived, the number of code changes submitted rose 3.1 times, while the number of reviewers rose 1.5 times. The share of changes getting a real human review fell from 39% to 21% (He and colleagues, a July 2026 preprint). If the number of reviewers doesn't grow with the work, a smaller share of it gets properly checked, and nobody decided that it should.
Checking AI output once it's live
People need criteria to judge AI output, but they work out the criteria by judging output, so the criteria shift with what they see (Shankar and colleagues, 2024, a preprint). The US government's voluntary AI risk framework from NIST sets out what good oversight looks like. The best practical template comes from an audit that found it missing. The inspector general for the US tax authority said the IRS needed defined measures, acceptable ranges for when a model drifts, and ongoing real-world testing (TIGTA, May 2025).
8. Measuring whether it paid
What to measure
Before you build, record five things about the old process. How long each case takes. How much hands-on time it takes. How often work is corrected or sent back. What each case costs. And what share goes through with no exception. My research found no study comparing these measures with each other. The principle that matters more is a comparison. A before-and-after against a rising market proves nothing. My paper on why AI projects fail shows how that played out at Zillow.
Count the time spent checking the AI as a cost. As the coding study above shows, volume can outrun review.
How long before the results mean anything
Longer than most plans allow. Deloitte surveyed firms across Europe and the Middle East and got 1,854 responses. By the respondents' own account, a typical AI use case took two to four years to earn a satisfactory return. Only 6% paid back within a year. The MIT report behind the "95% of AI pilots fail" headline concedes that its own six-month window "may be insufficient". In US Census manufacturing data from 2017 and 2021, firms' productivity dipped after they adopted AI before it improved (McElheran and colleagues, Census working paper CES-25-27, 2025). The authors say they don't estimate how long the dip lasts.
So at three and six months, look for direction and harm. Don't call success or failure on one quarter.
What the best studies found
| Study | Design | What it found |
|---|---|---|
| Brynjolfsson, Li and Raymond, QJE 2025 | Staggered roll-out, 5,172 support agents | +15% resolutions an hour on average; biggest gains for the less experienced |
| Noy and Zhang, Science, 2023 | Experiment, 453 professionals, writing tasks | 40% less time, 18% higher quality |
| Dell'Acqua and colleagues, 2026 | Trial registered in advance, 758 consultants | Gains on tasks AI is good at; worse answers on tasks it isn't |
| Cui, Demirer and colleagues | 3 field trials, 4,867 developers | 26% more tasks completed, with a wide margin of error; partly written by the vendor |
| METR, July 2025 | Trial, 16 experienced developers | Tasks took 19% longer with AI; developers believed it saved them 20% |
| METR, February 2026 | 57 developers, newer tools | Results not statistically reliable; METR thinks developers are "likely" faster now but its data is "very weak evidence for the size of this increase" |
| Dillon and colleagues (Microsoft) | Trial, 66 firms, 7,137 workers | 1.4 fewer hours a week on email across everyone offered it (about 2 for those who used it); no change in the mix of work |
| Humlum and Vestergaard, 2026 | 25,000 workers linked to Danish pay records | Any effect on earnings above 2% ruled out |
| UK Department for Business and Trade, 2025 | Evaluation, 1,000 licences, no counterfactual design | "did not find evidence that time savings have led to improved productivity"; productivity "was not a key aim", so little data was collected on it |
Two things come out of the table. First, people's sense of time saved isn't reliable: the METR developers believed the opposite of what the clock showed, and METR now calls that study out of date, but the point about self-report stands. Second, keep three things apart. Output per worker rose in several trials: support tickets resolved, developer tasks completed, sales in some retail uses. Pay didn't follow in the Danish records. And company-wide productivity hadn't moved for nine in ten executives surveyed. A gain on the task is not yet a gain for the business, and whether redesigning the work closes that gap hasn't been tested.
9. People and change
Adoption
In the US, 52% of workers are worried about AI's effect on their work and 36% are hopeful, and 55% rarely or never use AI chatbots at work (Pew Research, February 2025, 5,273 workers). In Denmark, giving workers information about AI's benefits changed what they believed but not whether they used it. Employer initiatives, such as encouraging use, nearly doubled it, from 47% to 83% (Humlum and Vestergaard, April 2025 version). Telling people doesn't move them. Encouraging them does.
What happens to jobs
The evidence on young workers is contested. One study using payroll data found that workers aged 22 to 25 in jobs exposed to AI were about 19% below where their earlier trend would put them (Brynjolfsson, Chandar and Chen, August 2026 update). But the authors say the patterns "attenuate when controlling for education", other US data show a much smaller gap, and the authors say it "does not estimate a causal impact of AI". Danish records rule out any large effect on young workers' share of jobs. Don't state this one as fact either way.
Losing skills
Doctors who used AI to help find growths during colonoscopies then found fewer on their own: the detection rate on procedures without AI fell from 28.4% to 22.4% (Budzyń and colleagues, Lancet Gastroenterology and Hepatology, 2025). It's one observational study. The authors say AI "might" reduce detection, and a commentary for the American College of Gastroenterology points to other possible causes. Headlines saying "AI deskills doctors" drop both caveats.
Reversals
Commonwealth Bank of Australia reversed 45 redundancies it had made after bringing in an AI voice bot, conceding that "this error meant the roles were not redundant" (as reported; there was no tribunal ruling). Klarna, which once said its AI did the work of 700 agents, later had its chief executive reported saying service quality fell, and that it was hiring people again. The 700 figure was a press claim and doesn't appear in Klarna's filing to list its shares.
10. Ten cases, checked by someone other than the vendor
These are deployments where the result was documented by a regulator, an auditor, a court, a government evaluation or a peer-reviewed study. They exclude cases already covered in my paper on why AI projects fail.
| # | Deployment | What it showed | Source |
|---|---|---|---|
| 1 | Customer support, US software firm | +15% resolutions an hour on average; biggest gains for the less experienced, small gains and a slight quality drop for the most skilled | Peer-reviewed, QJE 2025 |
| 2 | UCLA Health AI note-takers, 238 doctors | Best tool saved about 41 seconds a note, against about 18 in the comparison group; one tool showed no reliable effect; occasional significant inaccuracies | Trial, NEJM AI 2025, via UCLA's release |
| 3 | NHS stroke imaging AI, 26 hospitals compared with the rest of 107 | Clot-removal rates doubled at the AI hospitals (2.3% to 4.6%) but also rose 62.5% elsewhere, and many comparison hospitals took up AI on their own; at the AI hospitals, patients reviewed with AI were transferred faster (128 against 192 minutes) | Peer-reviewed; four authors work for the vendor and hold share options |
| 4 | NHS AI Diagnostic Fund, chest imaging, 66 trusts | Procurement and rollout "took longer than anticipated"; delays meant the first evaluation couldn't study effects on care or outcomes | Independent evaluation, NIHR 2025 (phase 1) |
| 5 | IRS audit selection | "has not established processes to evaluate whether the performance of AI models is better than prior methods"; evaluation was done before go-live, not after | Inspector general's audit, May 2025 |
| 6 | Australian Taxation Office, 43 models in use | Governance "partly effective"; 74% without a completed data-ethics assessment; "no evidence of structured and regular monitoring" of the models in production | Australian National Audit Office |
| 7 | UK Department for Business and Trade, Copilot, 1,000 licences | No evidence of a productivity gain; spreadsheet work slower and worse; 22% saw made-up answers | Government evaluation, August 2025 |
| 8 | Online retail platform, 7 AI uses tested at random | 4 raised sales, by up to 16.3%; the rest no detectable change | Preprint; one author worked for the platform, two consulted for it |
| 9 | Air Canada's chatbot | Gave a customer the wrong refund policy; the airline was held liable because it "did not take reasonable care" | Civil Resolution Tribunal, British Columbia, 2024 |
| 10 | Dutch childcare benefits | A risk model using nationality; a parliamentary inquiry found "fundamental principles of the rule of law" violated; a €2.75m fine; the cabinet resigned | Parliamentary inquiry 2020, data authority 2021 |
Australia's Robodebt scheme belongs alongside these as a warning, though it was rules-based data matching rather than AI. Its Royal Commission called it "crude and cruel" and unlawful (2023). I wrote about it in the piece on the human check that was cut.
The deployments that showed gains had been measured against a comparison. The two audits found no regular monitoring once the AI was live.
Read across, the cases point one way, with untidy edges. Cases 1, 2, 3 and 8 were measured against a comparison, and each found a real effect somewhere, alongside tools or uses that did nothing measurable. Cases 5 and 6 were audits, and both found that the AI wasn't being regularly monitored once it was live. Cases 4 and 7 were evaluations that couldn't answer the outcome question at all: one because of delays, the other because productivity wasn't what it set out to measure. That isn't proof that measuring causes success: firms that measure may be different in other ways. But my earlier failure paper found the same thing. Its two success stories had both set a measurable target in advance, one that could have shown the project failing.
11. The method: six steps for one process
Put the evidence together and you get a method. It works on one process at a time, where you can count the result. It isn't evidence for rolling AI out across a whole company at once.
Two things to know before using it. Most of the evidence sits on the checks, steps 1, 2, 5 and 6. And the evidence supports the shape of each check, not its settings: no study gives a universal number for how long to run a comparison or how many errors to plant. Those numbers are yours to set, which is why the worksheet at the end leaves them blank.
How much of it to use depends on what the process touches. For a low-risk internal task, a light version of each step is enough to start a controlled trial. For anything that affects customers, money or regulatory reporting, use all of it.
Steps 1, 2, 5 and 6 are the checks around the AI. That is where most of the evidence is.
1. Pick one process, one main number, and the things that mustn't get worse. Choose a single process and the person responsible for it. Choose one main number you want to improve, such as the cost of handling an invoice or the days it takes to settle a complaint. Then choose two or three guardrails: numbers that must not get worse, such as the error rate, complaints, or cases sent back. A process that gets faster while making more mistakes hasn't got better.
Write down what would make you stop, in terms of those guardrails, before you build anything. For example: "If the error rate is higher than the old process's, we switch it off." Research on IT projects that ran out of control found that clear criteria for success and failure helped people stop the projects that were failing (Keil and Robey, 1999).
Name two people: who decides whether it goes live or widens, and who can switch it off. The person whose target the AI is meant to hit shouldn't be the one who judges whether it worked. That's the point of what bank rules call "effective challenge": the check is done by someone objective, with no stake in the answer.
2. Measure how the process runs today. Record five things. How long each case takes from start to finish. How much hands-on time it takes. How often work is corrected or sent back. What each case costs. And what share goes through with no exception. Start with the numbers your systems already hold, check them against what people actually do (section 4 explains why they may not match), and give this stage a deadline.
Make sure you'll compare like with like. If the AI will take the easy cases and leave people the hard ones, the old process's average time is the wrong comparison. Measure the same mix of cases on both sides.
3. Decide who or what does each step. For each step, ask these questions in order. Can we cut it? Can we simplify it? Can it become a fixed rule? Can AI do it? Or must a person do it? Section 5 has the evidence for each.
Then design the exception path: where the cases the AI can't handle go, who deals with them, and how long they take. Hard cases that bounce out of an automated step can take people much longer than the routine work did, and their cost belongs in the comparison.
Where a mistake would be expensive or hard to undo, put a control on that step. A person approving it is one option. Others are limits on what the AI can do, a second independent check, or a way to reverse the action. A person with no time and no evidence in front of them protects very little.
4. Put the AI's answer in front of people at the moment they decide. Don't make anyone go and look for it in another screen or tool. That's a sensible design choice, supported by some of the hospital evidence and not by all of it (section 6). Show the evidence and how sure the AI is, not just its verdict. When someone overrides the AI, have them record why, and also review a sample of the cases where they agreed: if only disagreeing takes effort, agreeing becomes the easy path.
5. Test it against the old way, and test the people checking it. This step has two parts.
First, the comparison. Run the same kinds of cases both ways for a while, the old process and the AI version. The people working the old process shouldn't see the AI's answers. Someone independent of the project judges the cases where the two disagree. Run it long enough to cover your usual busy and quiet periods. Widen the AI version a piece at a time, and only when the main number is better and no guardrail is worse. Banks with approved internal models are expected to run this kind of comparison already (see below).
Second, the reviewer test. Airports measure their screeners by projecting fictional threats into real bags (section 7). You can do the same with the people checking the AI's work, but planting errors in real customer work is itself a risk. So run it in a copy of the work, never in live cases.
REVIEWER TEST: PLANTED ERRORS
WHERE A copy or shadow queue, never live work.
A planted case must not be able to pay,
send or post anything.
WHAT Realistic errors of the kinds that would
matter most, including the serious ones.
Written and checked by someone outside
the team being tested.
HOW MANY Enough to judge one person, not a
handful. Airport screening needed about
100 hard enough test images per screener
for a reliable individual score. With
less volume, track the team's catch
rate, not individuals'.
HOW HARD Hard enough that people miss some.
Easy tests make reviewers look better
than they are.
BLIND Reviewers know testing happens, not
which cases are tests.
SCORE Caught / flagged as unsure / missed,
weighted by how serious the error was.
STOP RULE Set before the first test: stop widening
if the catch rate falls below ____.
FEEDBACK Tell the reviewer about a miss straight
away, as airport systems do.
PEOPLE Agree it with staff and their
representatives first. Decide who sees
individual scores and what a low score
leads to.
CHECK IT Compare catch rates on planted errors
with real errors found later. If they
differ, fix the test.
6. Keep measuring after it goes live. Track the main number and the guardrails for as long as the AI is in use, and have someone independent of the owner look at them. At three and six months, look for direction and harm. Judge the payoff later, not on one quarter's figures (section 8).
Decide in advance what forces a re-test: the vendor changing or updating the model, a change to the prompts, data or tools it uses, or a shift in the kinds of cases coming in. A comparison passed six months ago tells you about a system you may no longer be running.
Keep the old way runnable. The process stays documented, and staff can still do the work without the AI: section 9 has evidence that skills fade when people stop using them. Practise switching it off before you need to.
Here is the method as a worksheet for one process. The blanks are yours.
ONE-PROCESS WORKSHEET
Process: ______________ Owner: ______________
Decides go-live and widening: __________ (not the owner)
Can switch it off: ______________
MAIN NUMBER ______________ today: ____ aim: ____
GUARDRAILS 1 ______________ today: ____
(must not 2 ______________ today: ____
get worse) 3 ______________ today: ____
STOP RULE Stop if ________________________________
BASELINE from ____ to ____ cases: ____
same case mix the AI will get? Y / N
EXCEPTIONS where hard cases go: ______________
who handles them: ______________
time per exception: ____
COMPARISON cases run both ways: ____ for ____ weeks
disagreements judged by: __________
REVIEWER TEST shadow queue only? Y / N
planted per reviewer per period: ____
written by: ______________
stop if catch rate falls below: ____
agreed with staff on: ____
RE-TEST WHEN model or vendor change / prompt, data
or tool change / case-mix shift /
otherwise every ____ months
FALLBACK old process documented? Y / N
staff can still do it unaided? Y / N
last switch-off practice: ____
Banks with approved models already run most of this
Steps 5 and 6 are what the audits found missing. In the UK, the Prudential Regulation Authority's supervisory statement SS1/23 applies to banks approved to use their own models to set capital. It expects them to compare each model with "challenger models" and to run "parallel outcomes analysis". It also expects them to monitor vendors' models and to name a senior manager accountable for model risk. It covers AI used in those firms' models. Banks without that approval aren't bound by it, though the regulator says they "may find the proposed principles useful". Its April 2026 update was a clarification: the expectations "are not conditions for internal model approval", and the regulator said it was "clarifying the existing policy, and not changing it".
UK banks with approved internal models are expected to run these checks, AI included. The current US guidance leaves generative AI out.
In the US, the April 2026 guidance that replaced SR 11-7 says generative and agentic AI models "are not within the scope of this guidance", and expects to be most relevant to banks with over $30 billion in assets. It tells banks to decide governance and controls for systems it doesn't cover through their own risk management. So on generative AI, the UK expectations are now the more explicit of the two.
The AI firms' job adverts show what's missing. None of the engineering adverts in my research mentions a challenge function: the people whose job is to try to prove the model wrong. Bank model-risk practice has one. Whether buyers outside banking will pay for it is a question this research can't answer.
12. Repeated most, supported least
You'll hear these in nearly every pitch.
- "70% of change programmes fail." It began as an "unscientific estimate" about re-engineering, and a review found no reliable evidence for it. Section 4.
- "95% of AI pilots fail." The report it comes from concedes its own window was too short. My paper on why AI projects fail traces it.
- "Forward deployed engineer postings are up 800%." The trail ends at a newspaper citing a job-listings site, and neither could be opened. Section 3.
- "Klarna's AI does the work of 700 agents." Klarna's own press release said it did "the equivalent work of 700 full-time agents"; the figure isn't in its share filing, and its chief executive was later reported saying quality fell. Section 9.
- "Buying beats building, 67% to 33%." That's how often projects reached deployment, from interviews with 52 organisations, and the authors say it may reflect capability, not approach. Section 3.
- "The firms that get value redesign their workflows." Correlations, mostly from consultancies that sell the redesign. No randomised test exists. Section 2.
- "Put a human in the loop and it's safe." The research on automation bias says otherwise, and the EU AI Act names automation bias itself. Section 7.
- "Build it into the existing system and people will use it." The two big reviews of hospital trials disagree on this. Section 6.
Across this research, the popular versions of claims kept turning out cleaner than their sources: the caveat the authors stated was the first thing to go. Peer review of this paper caught it doing the same in a few places, now fixed.
13. Limits, and what would prove this wrong
The method is put together from other fields. Steps four to six come from clinical decision support, airport screening and bank model risk, not from trials of AI deployment, because almost none exist. Applying them to AI is reasonable, but it's a judgement, not a tested result. Several key studies are from one firm, not yet peer-reviewed, use a simulated AI, or were written by the vendor, and each is marked as such above.
The method doesn't fit company-wide bets on a single AI platform, where there's no one process to measure. It struggles with low-volume work that's mostly judgement, where there's nothing to count. And it may not hold for AI agents acting across many systems at speed, where having a person approve every action may not be possible.
The fair objection from anyone running a programme is cost. A comparison means running two processes for a while. Planting errors and tracking reviewers is work that has to go in the budget, and a comparison can turn up work nobody planned for, such as a backlog of exceptions. The checks aren't free, and their cost isn't fully known in advance. The cost of a deployment that wasn't measured is also real, and it turns up later, in an audit report, as it did at the IRS and the Australian Taxation Office.
A second objection is about who runs the checks. In a bank, independent model validators, a second line of defence and a regulator stand behind them. In most other organisations, the person who chose the process may also run the test and judge the result. The method asks for an independent decider and checker in steps 1 and 6 for that reason, but it can't supply the independence. That's an ownership question each organisation has to answer first.
Two things would prove the emphasis on checks wrong. A randomised study across firms showing that deployments with no baseline, no comparison and no measured review pay off as often as those with them. Or official data from 2026 and 2027 showing firm-level gains from staff simply picking the tools up, with no redesign.
Three questions this research couldn't close:
- Has anyone measured the results of running AI alongside an old process?
- Are planted errors used to test reviewers in audit or finance?
- How well do interviews match what office workers actually do?
Articles from this paper
Shorter pieces that each take one part of this paper further. New ones are added here as they're published.
- The usual signs that AI is working can mislead: six ways organisations judge whether AI is working, why each can mislead, and the tests hospitals, airports and banks already run. Sections 7, 8 and 11.
- The engineers AI firms send into companies, and their method: one firm's method for putting AI into a process, and the questions it left open. Sections 3 and 4.
- Why AI projects fail: 19 headline statistics, and what each one actually counts: the earlier paper, on the failure rates. Sections 2 and 12.
- If AI saves you time, why aren't you being paid more for it?: what the Danish pay records show. Section 8.
- The human check that was cut: Australia's Robodebt scheme. Section 10.
- An AI company wants to buy your firm? Ask it to prove the AI works first: the same tests, for owners being approached by AI roll-ups. Section 11.
Sources
How this was produced: a research run on 13 October 2026 that pulled primary sources and graded each claim by the strength of its evidence: trials, audits, filings and official records first; surveys and working papers next; press reports and vendor statements last. The full evidence file, with every source's grade and caveats, is in my research notes.
Adoption, productivity and pay: US Census Bureau, business AI use survey, 2025-26; Bonney and colleagues, CES-WP-26-25, April 2026; Yotzov, Barrero, Bloom, Davis and colleagues, NBER w34836, 2026; Baslandze and colleagues, NBER w34984, March 2026; Bick, Blandin, Deming and Schumacher, NBER w35677, 2026; Humlum and Vestergaard, BFI working paper 2025-56 and revision of March 2026; IZA discussion paper 16992, 2024; Noy and Zhang, Science, 2023; Cui, Demirer and colleagues, Management Science; Dell'Acqua and colleagues, Organization Science, 2026; Brynjolfsson, Li and Raymond, Quarterly Journal of Economics, 2025; Dillon and colleagues, arXiv 2504.11436; Otis and colleagues, Management Science, 2026 (formerly HBS working paper 24-042); Brynjolfsson, Chandar and Chen, 2026 update; Fang and colleagues, arXiv 2510.12049; He and colleagues, arXiv 2607.01904, 2026; McElheran, Yang, Kroff and Brynjolfsson, CES-25-27, 2025.
Independent research: METR, randomised trial of experienced open-source developers, July 2025, https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ , and February 2026 update; Pew Research Center, February 2025.
Government evaluations and audits: UK Department for Business and Trade, Microsoft 365 Copilot evaluation, August 2025; Government Digital Service, cross-government Copilot experiment, June 2025; Treasury Inspector General for Tax Administration, report 2025-308-022, May 2025; Australian National Audit Office, governance of AI at the Australian Taxation Office; NIHR evaluation of the NHS AI Diagnostic Fund, 2025.
Firm statements, adverts and filings: OpenAI, Deployment Company announcement, 11 May 2026; Goldman Sachs Asset Management, release on the Anthropic services venture, 4 May 2026 (Brad Lightcap's quote is from press reporting); Anthropic, OpenAI and Palantir job adverts, October 2026; Palantir S-1 (2020) and 2025 annual report; Salesforce FY26 annual report and results; Accenture FY25 and FY26 results; ServiceNow FY24 annual report; Klarna F-1; Contracts Finder and NHS England on the Federated Data Platform; Australian Greens release and Accounting Times on the Deloitte refund; Kirkpatrick, Sturdy and colleagues, Policy & Politics, 2019; Sturdy and colleagues, Public Administration, 2022; Microsoft and LinkedIn, 2024 Work Trend Index; Klarna press release, 27 February 2024; Deloitte, "AI ROI: The paradox of rising investment and elusive returns", 22 October 2025; BCG, "The Widening AI Value Gap", September 2025; MIT NANDA report.
Finding the real process: IEEE Process Mining Manifesto, 2011; Bose, Mans and van der Aalst, 2013; Benevento and colleagues, arXiv 2209.10897; Mans and colleagues, 2008; Alter, 2014; Suchman, 1983; Hollnagel, 2017; Dale and colleagues, 2010; GOV.UK Service Manual, discovery phase; Hammer, Harvard Business Review, 1990; Hammer and Champy, 1993, via secondary sources; Davenport, 1995; Hughes, 2011.
Deciding what AI does, and building it: Wewerka and Reichert, 2020; Parasuraman, Sheridan and Wickens, 2000; Onnasch and colleagues, 2014; EU AI Act, Article 14; Federal Reserve SR 11-7, 2011, and its replacement SR 26-2 / OCC Bulletin 2026-13 with the revised interagency guidance, 17 April 2026; Prudential Regulation Authority SS1/23, https://www.bankofengland.co.uk/prudential-regulation/publication/2023/may/model-risk-management-principles-for-banks-ss ; PRA, Low Impact Amendments Finalisation (LIAF01/26), 23 April 2026; Financial Conduct Authority, AI update, 2024; Kawamoto and colleagues, BMJ, 2005; Roshanov and colleagues, BMJ, 2013; Goodhue and Thompson, 1995; Beede and colleagues, 2020.
Testing and checking: Google SRE Workbook, canarying releases; EU Regulation 2015/1998, Annex, Division 12; Buser and colleagues, Transportation Research Part A, 2025; Wolfe and colleagues, Nature, 2005; Evans, Birdwell and Wolfe, 2013; Just and colleagues, 2014; Parasuraman and Manzey, 2010; Goddard and colleagues, 2012; Mosier, Skitka and colleagues, 1996; Dratsch and colleagues, Radiology, 2023; Bansal and colleagues, 2021; Buçinca and colleagues, 2021; Kim and colleagues, 2024; Green, 2022; Shankar and colleagues, 2024; NIST AI Risk Management Framework and AI 600-1.
Cases and people: Budzyń and colleagues, Lancet Gastroenterology and Hepatology, 2025, and the ACG commentary; UCLA Health release on the ambient AI scribe trial (NEJM AI, 2025); Nagaratnam and colleagues, Lancet Digital Health, 2025; Moffatt v Air Canada, 2024 BCCRT 149; Dutch parliamentary inquiry "Ongekend onrecht", 2020; Robodebt Royal Commission, 2023; reporting on the Commonwealth Bank reversal and on Klarna's chief executive.