Make each rule a yes or no: why half your saved instructions stopped firing

Overview

Why now. One benchmark stacked 500 separate “use this word” rules into a single prompt and found the best frontier models still landed 68% of them. Another, from an unrelated team, gave models a handful of rules at once and watched the odds of getting all of them fall off a cliff. Both results are real, and they do not contradict each other — they were measuring different shapes of rule. So if your saved instructions have quietly stopped firing, the problem is probably not how many you wrote. Rewrite each one as something you could tick as present or absent in the finished text, and take the word-count and keep-it-consistent-throughout rules out of the saved list.

The content

You have an instruction set somewhere that you no longer read: Project instructions, a Gem, a custom GPT, an instruction file in your work assistant, or a block of text you paste at the top of every chat. It has accumulated. A tone rule, a formatting rule, a list of banned words, a citation rule, and somewhere near the bottom, “keep it under 400 words”. Each one behaved when you tested it on its own. Together, half of them don’t.

The standard advice here is to shorten the list. That is not wrong, but it aims at the wrong variable. Rules are not interchangeable units that each cost the same to add. Some stack almost for free. Others fall over almost immediately — and you can tell which is which by reading the rule, not by counting the list.

The tick test. Could someone hold your finished text and mark this rule present or absent, without reading the whole thing and keeping a running tally? “Ends with a sign-off” — yes. “Never uses the word ‘leverage’” — yes. “All lowercase” — yes. “Under 400 words” — no; you have to count. “Every section about the same length” — no. “Consistent heading style throughout” — no.

The rules that fail the tick test are the ones that make the model hold a running state while it writes, and those are the first to go. A benchmark posted on 12 August measured this across 15 models and roughly 370,000 automated checks: a word-count rule and a paragraph-count rule each fell below half their solo reliability once about five rules were stacked together, while a “must include these words” rule was still above half at twelve. A separate team, testing a different way, summed up the same asymmetry in one line: “Lexical survives; format and length collapse hardest.” Their fastest-collapsing rules included exactly five bullets, begin with “Answer:”, and return valid JSON. Their most durable were no question marks, avoid “however”, and all lowercase.

Here is the part that turns this from a fact into a habit: the tracking rules fail quietly. You do not get “I couldn’t hold that one.” A study across 26 models named a model’s sense of how long its own output is running as one of the causes of the trouble — it writes to an impression of the length rather than a count. So the failure reaches you as a confident 620-word answer that reads as though someone believed it was 400.

This is also not a small-model story. The August benchmark’s line-up included the current frontier; even the strongest model was under water by seven simultaneous rules, twelve of the fifteen broke at three or fewer, and the rankings inverted along the way — the models that handled constraints best one at a time were not the ones that combined them most reliably. The paper’s own summary is that “reliable instruction following breaks down beyond 5-6 simultaneous constraints”. Take the direction as well supported and the exact number as unsettled: both papers are still preprints, one of them reviewed in an ACL Rolling Review cycle and neither published, and the boundary moves with the model.

Try it

Ten minutes, on the instruction set you actually use.

1. Open it and read it properly — the Project instructions, Gem, custom GPT, assistant instruction file, or the block you paste each time. Most people have not looked since they wrote it. (Check what your workspace has enabled: if saved instructions are switched off for you, the pasted block is the same artefact and this exercise is identical.)
2. Tick-test every line. Present-or-absent, or requires-a-tally?
3. Keep the ticks, move the tallies. A tallying rule comes out of the standing set and goes into the one request that genuinely needs it — where it is competing with two other rules instead of nine.
4. Make length a second step. Instead of a standing “under 400 words”, let it write, then ask: “Now bring this to about 400 words, keeping the second and fourth points.” That converts a tracking rule into a single job.
5. Check your work. Give the trimmed set one real task and see which rules actually fired.

Where it breaks. Present-or-absent is a better bet, not a guarantee. A 2026 analysis found that precise formatting rules — including “end with this sign-off” — did modestly worse when small open-weight models were run in extended-thinking modes, so a tick-test rule can still fail for reasons that have nothing to do with stacking. And the tick test tells you only whether a rule will survive company. It has no opinion on whether the rule was a good idea.

Additional reading

• LLMs Can Follow Instructions, But Not Many at Once — arXiv:2608.12426 — the per-rule half-lives and the frontier line-up. Preprint (reviewed in an ACL Rolling Review cycle, unpublished).
• Instruction Stacking Collapse — arXiv:2608.02639 — an independent benchmark reaching the same split by category. Preprint; three models.
• IFScale — arXiv:2507.11538 — 500 keyword rules at once — how far binary rules stretch before even they fray.
• LIFEBench — arXiv:2505.16234 — why length rules fail without announcing it.
• Constraint-type asymmetry under reasoning — arXiv:2606.09662 — the caveat, measured on small open-weight models: precise formatting rules can degrade under extended thinking.

Editor’s note

These rules will continue to develop. I find myself constantly tweaking my general and task-specific instructions to try and manage issues, and move with model developments and lab findings. It’s an imperfect science, but a rule you can check without counting is a sound target for now. Give it a go when you notice things starting to slip.