# The AI Eloquence Trap

Published: 2026-07-22
Canonical: https://polyviam.com/writings/ai-eloquence-trap/
Excerpt: AI is getting better at sounding right faster than it's getting better at being right. Here's why fluency fools us, and four defences against it.

AI eloquence is increasingly a problem. Models are getting better at sounding smart faster than they are getting better at being right.

A model delivers a wrong answer in exactly the same fluent, confident, well-structured prose as a right one. AI researchers have a name for it: the eloquence trap. We are hard-wired as humans to read "easy to process" as "probably true."

What's more, AI models are unquestionably getting better at solving hard problems: PhD-level science, deductive puzzles, maths. But at the same time, reasoning models consistently underperform plain counterparts on the easy end of the distribution, the overthinking problem you may have experienced yourself.

When Apple researchers ran reasoning models through controlled puzzles, they found that on the simpler problems, the models that "reasoned" more scored worse than the plain ones ([Shojaee et al., 2025](https://machinelearning.apple.com/research/illusion-of-thinking)). They'd locate the answer, then keep going: exploring dead ends, adding steps, reasoning themselves back out of a correct solution. A team at the University of Maryland measured the same thing and called it an "illusion of improved reasoning" ([Ghosal et al., 2025](https://arxiv.org/abs/2506.04210)). It raises the variance of the output while lowering its accuracy.

> More thinking, less right.

Put these two things together, eloquence and overthinking, and you have a compounding issue. But overthinking is just one way the machine arrives at a wrong answer. **Eloquence is why you don't notice.**

In a 2026 experiment on visual reasoning tasks, the kind where you're checking whether the model read the pattern correctly rather than following its language, people shown a fluent explanation caught the AI's mistakes only 16% of the time. People shown the model's plain confidence numbers instead caught it 38%, more than double ([Cohen et al., 2026](https://arxiv.org/abs/2604.03237)).[1](#fn1) The explanation didn't help them think. It stopped them noticing. Misdirection, like a magician.

I catch myself doing this. I read a beautifully argued case for something, three steps of reasoning, caveats handled, and only later realise the honest answer was one sentence long. The elaborate version wasn't more right, just more overthought.

Plot it on a 2x2 and the trap is the top-left corner: answers that sound brilliant and are wrong. Overthinking pushes answers toward "wrong." Eloquence keeps them all sounding "brilliant." The two aren't independent; they aim at the same corner.

![A 2x2 matrix plotting AI answers on two axes: sounds brilliant vs sounds flat, and wrong vs right. Overthinking pushes answers toward wrong; eloquence keeps them sounding brilliant. The danger zone is the top-left quadrant: sounds brilliant, wrong.](/img/writings/ai-eloquence-trap-quadrant.svg)

_The trap vector: overthinking pushes answers toward wrong, eloquence keeps them sounding brilliant._

Eighteen-plus months ago it was easier to identify when a model was wrong; the writing wobbled. Now the writing almost never wobbles. Not much sits below the "brilliant" line anymore.

So what do you do about it? There are four defences.

## 4 defences to the eloquence trap

**Know the subject yourself.** If you're genuinely deep in a field, the polish doesn't land; you read the elaborate answer and think, "no, it's simpler than that." Studies on AI-assisted medical diagnosis show it cleanly: high-expertise clinicians dismiss bad AI advice, low-expertise ones defer to it. But nobody is an expert in more than a sliver of what they touch. Everywhere else, you're the one being impressed.

**Prompt for facts and confidence, not an essay.** In that same study, people shown the model's raw confidence caught its errors more than twice as often as people handed a polished explanation. The prose persuades; the number informs.

**Ask it to steelman the other side.** "Now give me the strongest case that this is wrong." The doubt the fluency papers over is usually still in there; you just have to ask for it.

**Match the model to the problem, not to the stakes.** The instinct is to reach for the heaviest reasoning model whenever something matters. But a simple factual question doesn't need deliberation, it needs recall, and the reasoning model is the one most likely to overthink it into a confident mistake. Save the deliberation for problems that actually have depth. Right now the tools won't make this call for you. That's your job, and it's becoming a real skill.

None of this makes the model more correct. You can't control that. It makes you harder to fool. Right now, that's the more valuable skill.

[^1]: Worth being precise about where this holds: on language-based reasoning tasks, the same researchers found explanations helped rather than hurt. The danger zone is specifically where you can't independently verify the model's read at a glance, and default to trusting its account of itself instead. [↩](#fnref1)