Does 'Act as a senior lawyer' actually help? The truth about role prompting

Overview

This module interrogates one of the most-repeated lines in prompting advice: open with “Act as a senior lawyer” and the model gets smarter. In December 2025, Wharton’s Generative AI Labs published a controlled study — “Playing Pretend” — testing expert personas across six models on graduate-level questions, and found they did not reliably improve accuracy; some low-knowledge personas made it worse. By the end you’ll know what role prompting actually does, what it doesn’t, and where to spend your prompt instead.

The content

The obvious read is intuitive: tell the model to be an expert and it will reason like one. So we all paste “You are a world-class M&A lawyer” at the top and assume the facts that follow are now more trustworthy.

The evidence overturns that. The Wharton team tested in-domain expert personas (a physics expert on physics questions), off-domain experts, and low-knowledge personas across GPQA Diamond and MMLU-Pro. Across both benchmarks, persona prompts generally did not beat a plain no-persona baseline. Matched experts had no significant impact on most models; low-knowledge personas like “layperson” or “toddler” were generally harmful, and a domain-mismatched persona made one model start refusing to answer at all. This is not a one-off result — a peer-reviewed 2024 study ran 162 personas over 2,410 factual questions across four model families and reached the same conclusion: adding a persona did not improve performance versus no persona.

Here is the distinction that matters. Roles steer tone, not truth. Telling the model it’s a lawyer reliably changes how it writes — register, structure, what it foregrounds — but it does not reliably change whether the law it states is correct. As Lennart Meincke, a research fellow at Wharton’s Mack Institute, told Penn Today, “people assumed personas would really help, but it matters less today” with capable frontier models. The persona is a style dial, and we have been mistaking it for an accuracy dial.

The failure mode is the credible part. A confident “senior lawyer” voice makes a wrong answer more persuasive, not less — you have dressed a hallucination in a suit. The fix is not a better job title. It’s the unglamorous work the persona was distracting you from: give the model the actual source material to reason over, write specific task instructions and constraints, and verify the output against something real. Keep the role line if you want the output to read like a lawyer wrote it. Just don’t let it stand in for grounding.

Try it

Next time you reach for “Act as a [senior expert]”, split the job. Keep the role for voice if you want it, but move the real weight onto context and verification. On your own current task, try this structure instead of leading with a persona:

Task: [the specific thing you need done, with the decision it feeds]
Source material: [paste the actual document / data / policy — do not rely on the model's memory]
Constraints: [format, length, what to exclude, what "good" looks like]
Instructions: Answer only from the source material above. For any claim not
supported by it, say "not in the provided material" rather than guessing.
Then list any point a reviewer should independently check.

Where this won’t save you: if the answer genuinely isn’t in the model’s training or your pasted context — a niche jurisdiction’s current rule, last week’s pricing — no role and no instruction conjures it. Persona or not, that’s a retrieval problem, and the honest move is to go and get the source yourself.

Additional reading

• Playing Pretend: Expert Personas Don’t Improve Factual Accuracy (Prompting Science Report 4) - Wharton Generative AI Labs — the December 2025 study behind this module: six models, GPQA Diamond and MMLU-Pro, in-domain, off-domain and low-knowledge personas.
• Why you shouldn’t ask chatbots to act like an expert - Penn Today — plain-language writeup with the researchers’ framing on why personas matter less now and what to do instead.
• When ‘A Helpful Assistant’ Is Not Really Helpful (Findings of EMNLP 2024) — earlier peer-reviewed study: 162 personas, 2,410 factual questions, four model families, no improvement over no persona.

Editor’s note

“You are a senior lawyer” is a formatting instruction, not a guarantee of substance — and a persuasive register makes a wrong reading easier to approve, at a cost that can be real. I also wonder about the metric we bolt on for authority: “an award-winning architect with 20 years’ experience.” Fine — so shall we omit anything contributed by people born after 1986? Keep the persona if it’s tone you’re after; put the real effort into the source and the verification.