The verification divide: why the same AI helps the best and hurts the rest

Overview

Give the same AI adviser to hundreds of Kenyan small-business owners and watch what happens. In the randomised trial written up by MIT Sloan Management Review this April, owners who were already performing well grew revenue and profit by around 15%. The strugglers — the people the tool should have helped most — went backwards by roughly 10%. Same tool. Opposite outcomes.

A formal model accepted at ICML 2026 argues that pattern is structural, not an accident of one experiment. The line between the two groups isn’t prompt skill, seniority, or effort — it’s whether you can catch the AI’s errors. And the divide isn’t a gentle slope; the theory says small differences in that one ability produce sharply different outcomes.

The content

The comfortable read of AI at work is a rising tide: everyone gains, some more than others. The evidence from judgment-heavy work says something harsher — and more useful.

The Yale and Nanjing researchers behind the ICML paper modelled the choice every AI user makes on every task: how much to delegate, and how much effort to spend checking the output. Their result is what they call a phase transition — “arbitrarily small differences in verification ability lead to sharply different behaviors.” People who can reliably catch AI errors delegate confidently and get amplified. People who can’t don’t just miss mistakes — they rationally check less and delegate more, because given their real ability, that is their best available move. Their work quality degrades “even when baseline task success improves and no behavioral biases are present.” Read that carefully: it isn’t laziness or gullibility. Call it the verification divide — and note that the fix it points to isn’t willpower, it’s raising the one ability the divide turns on.

The Kenyan field experiment shows the mechanism in the wild. In the researchers’ words, “weaker performers followed generic or misleading AI advice because they lacked the judgment to filter it out.” The advice looked identical on both sides of the divide; only one side could tell which parts of it were wrong.

Here’s the honest boundary, because who benefits flips with the kind of work. In Alibaba’s customer-service field experiment, low performers gained most and the gap narrowed — while top performers actually slipped, largely because the assistant nudged them into multitasking. Across three corporate coding trials totalling 4,867 developers (Management Science, February 2026), completed tasks rose about 26% and the less experienced developers adopted more and gained more. On well-scoped work with fast feedback, AI lifts the floor. The divide bites where work is ambiguous and wrong answers don’t announce themselves — advice, analysis, drafting, judgment calls. That’s most knowledge work.

Two more findings complete the picture. First, verification is measurable and trainable, not a fixed trait: a validated 13-item scale published in December breaks it into three factors — verification behaviour, motivation to check, and reflection — and higher scores predicted actually catching falsehoods in a live AI fact-checking task, across six studies with 1,341 participants. Second, you can’t hand the job back to the machine. Models can’t reliably self-correct their own reasoning — performance sometimes gets worse after self-correction (ICLR 2024) — and a July 2026 benchmark of 12,326 failed agent runs found the best model could locate the step where an agent went wrong only 73.9% of the time on text tasks, collapsing from 94% on short runs to 50% on long ones. The checking, for now, is yours.

Try it

Before your next real AI delegation — a brief, an analysis, a draft for a client — write the definition of done first, as three checks the output must pass:

Done means:
1. <a figure that must reconcile with a source you name>
2. <a claim that must trace to a document that exists and says it>
3. <a constraint that must hold — scope, date, jurisdiction, format>

Then run the task and apply the three checks yourself. Not “are you sure?” — asking the model to double-check itself is not verification.

Where it breaks is the point: if you can’t write three concrete checks, you’ve found the tell that you’re on the wrong side of the divide for this task. Don’t delegate it yet — do it by hand once to learn what good looks like, or shrink the delegation until you can name the checks. This runs in any tool your workplace gives you; it’s a habit, not a feature.

Additional reading

• Delegation and Verification Under AI — arXiv (v2 25 June 2026; ICML 2026) — the formal model: phase transitions in delegation behaviour; weak verifiers rationally over-delegate and reduce oversight, degrading their work quality even as baseline task success improves. Theory, not an experiment — read it as the structural explanation the field results keep matching.
• How AI Helps the Best and Hurts the Rest — MIT Sloan Management Review (20 April 2026) — the Kenya RCT write-up: GPT-4 as a WhatsApp business adviser for hundreds of small-business owners; roughly +15% for high performers, nearly −10% for strugglers who couldn’t filter bad advice.
• Generative AI in Action: Field Experimental Evidence from Alibaba’s Customer Service Operations — arXiv (8 February 2026) — the floor-lifting case: low performers gained most in speed and quality; top performers saw quality dip amid multitasking; customer ratings rose while objective retrial rates didn’t move.
• The Effects of Generative AI on High-Skilled Work — Management Science (online 27 February 2026) — RCTs at Microsoft, Accenture and a Fortune 100 firm, 4,867 developers: ~26% more completed tasks, with less experienced developers adopting more and gaining more. The counterweight: on well-scoped work, AI lifts the floor.
• The Critical Thinking in AI Use Scale — arXiv (13 December 2025; v2 23 July 2026; since published in Computers in Human Behavior Reports 22, 2026) — six studies, N=1,341: a validated 13-item measure with three factors (Verification, Motivation, Reflection); higher scores predicted more diverse checking strategies and better accuracy in a live fact-checking task.
• Who&When Pro: Can LLMs Really Attribute Failures in AI Agents? — arXiv (10 July 2026) — 12,326 failed agent trajectories across 26 benchmarks: best model finds the failing step 73.9% of the time on text (error-mode F1 only 22.2%), falling from 94% on short traces to 50% past 12K tokens; models label failures by surface similarity rather than root cause.
• Large Language Models Cannot Self-Correct Reasoning Yet — arXiv (ICLR 2024) — without external feedback, models struggle to self-correct and sometimes get worse; why “check your answer” prompts are not a verification strategy.
• Management as AI Superpower — One Useful Thing (27 January 2026) — Mollick’s delegation framing, including the question this module’s Try-it operationalises: “what should you check before telling me you’re finished?”

Editor’s note

In contract review I hand first passes to a model freely, because a mischaracterised clause can often feel wrong to me before I have fully formed the articulation of why. In areas where I have no such reflex, the model’s confidence and mine are indistinguishable. This is the danger zone. If I can’t grade the work, then usually the task that I’m handing over is not the right task.