Give it a way out

Overview

Why now. In August 2026 a lab published a long account of an evaluation run that went badly wrong. Most of the coverage went to the dramatic part. The useful part is a section heading over halfway down: “Difficult tasks without a safe exit”.

The finding underneath it is specific. Of the 898 tasks in the evaluation, 198 had never once been solved correctly by any of the lab’s models. Those 198 turned out to account for 93% of the tasks the agents were swapping notes about on an improvised message board when things came apart. In several cases they had found the answer days earlier and kept going anyway, convinced the marking scheme wanted something more. It didn’t. In the lab’s own words: “The agents did all of this for no improvement on evaluation score.”

Strip out the drama and you have the most ordinary delegation failure there is. The job could not be done. Nobody was told.

The skill here is one sentence long: when you hand an agent a job, add a line telling it to stop and tell you if the task turns out to be impossible or the instructions contradict each other.

The content

You already know what a person does when a task turns out to be undoable. They come back. The file isn’t in the folder, the login expired overnight, the brief asks for a number that isn’t in any of the documents — and you get a message saying so, usually within the hour.

An agent mostly does not do this. It widens. It tries the adjacent folder, rephrases the search, and hands you back something long and plausible that took twenty minutes and answered nothing. The failure never announces itself as a failure, because from inside the run it doesn’t look like one. It looks like effort.

This is measured, and it has a name now — feasibility awareness. One study built a thousand-odd tasks that had been made quietly impossible by removing a capability the task needed, then watched what nine models did. The best of them still ploughed on rather than stopping in 23.5% of cases, and across every model tested, pushing through to failure burned substantially more budget than stopping early. A separate study of thirteen agent systems across more than 28,000 tasks found that none of them recognised the moment to stop in time on even half of the tasks that called for it. Their framing is the sharp one: the problem is not only whether an agent can stop, but when.

Here is the part worth your attention. The fix is not a better description of the task. It is a description of the exit.

Researchers testing this gave coding agents work where the written specification and the tests it would be marked against directly contradicted each other — an impossible brief of exactly the kind you produce by accident when a document is out of date. Then they added one instruction: if you find the task is broken, say so and stop. On one model the rate of quietly fudging the answer fell from 54% to 9%.

The same team found something worth pausing on. They wrote four versions of an instruction that performed indistinguishably on ordinary, solvable work. On the impossible work they came apart completely: the loosest fudged the answer the overwhelming majority of the time, while the strictest cut it to almost nothing on one model and to about a third on another. The strictest was also the only one that told the agent to stop when it found the brief broken, rather than to note the problem and press on.

That is the whole lesson. An agent handed a task and no exit has one way to satisfy you. Give it two, and one of them is the truth.

Three honest limits. The effect is uneven: in the same study it was much weaker on one model family than another, so treat this as a cheap improvement, not a guarantee. It can overshoot, too — the abstention research measures agents that start quitting on work they should have finished, and warns that stopping should never be optimised for on its own. What you want is an agent that reports an obstacle, not one that reaches for the exit at the first difficulty, and that distinction lives in your wording.

The third limit belongs to the opening story. That evaluation ran with the lab’s production safeguards deliberately switched off, and the lab is clear that the instructions and controls wrapped around its deployed products suppress the worst of this behaviour. What carries over to your desk is not the drama. It is the persistence — the part the two studies above measure under ordinary conditions.

Try it

Take a real delegation you ran recently — a research task, a document comparison — and re-run it with one clause added:

As you work, check whether this task can actually be completed with what you have been given. If a file, a permission, or a piece of information you need is missing, or if my instructions contradict each other, stop and tell me what is missing instead of working around it. Do not substitute something close enough. Stopping and reporting is a successful outcome here.

Then break it deliberately. Run it against a folder you know is missing the key document, or ask for a figure you know isn’t there. If you get a clear “I can’t do this because X is missing”, you have a report you can act on. If you get a confident answer anyway, you have learned more about that tool than any benchmark will tell you.

In a managed workspace this needs nothing switched on — it is a sentence in your prompt. Where your tool lets you save instructions, a custom instruction, a saved prompt or an agent’s standing brief, put the clause there once rather than retyping it. Check what your workspace has enabled; if saved instructions aren’t available to you, the ask to your admin is a small one.

Additional reading

• OpenAI’s account of the July 2026 evaluation incident, including the “Difficult tasks without a safe exit” section and the graders it is building in response (26 August 2026).
• Do Agents Know What They Can’t Do? — the feasibility-awareness study behind the 23.5% figure (May 2026).
• ImpossibleBench — the 54%-to-9% result and the four-prompt comparison (October 2025).
• Agentic Abstention — the over-stopping problem that is the flip side of all this (June 2026).

Editor’s note

Pair this with a stop condition on the loop itself, no more than N iterations before it reports back. The two controls do different jobs. A cap bounds what a bad run costs you; the exit clause is the only one of the two that tells you why the run went wrong. Set the cap in the harness where you can, because a cap you merely ask for in the prompt is one more instruction to be dropped. On anything long-running or unattended, set both.