Confident is not correct

An AI answer arrives in tidy paragraphs, with no pauses, no "let me check" and no sign of strain. That polish is the trap. Most of us learned to read smooth, confident writing as a sign that the writer knows the subject. With a person that is often a fair guess. With AI it is not, because it writes a wrong answer in exactly the same voice as a right one.

Why polish fools you

Sounding right and being right are two different skills. The AI is very good at the first one: it has read a vast amount of well-written text and can produce more of it on any subject. Whether the facts inside are true is a separate question, and nothing in the tone tells you the answer.

Asking it how sure it is doesn't settle it either. Researchers who tested this found that models tend to be overconfident when they put their confidence into words. So "I'm 95% sure" from a chat assistant is a way of speaking, not a measurement. Don't use it to decide whether to check.

Hedges, on the other hand, are useful. In a large pre-registered experiment, people who saw answers that opened with "I'm not sure, but..." went along with wrong answers less often and got more questions right overall. A hedge is the AI handing you a reason to look closer. Treat it as information, not as weakness.

This picture was made by AI. Here's how to tell.

Pictures fool people the same way words do. At a glance this looks like a phone photo of someone checking a cafe bill. It isn't. An AI image model, FLUX.1 schnell from Black Forest Labs, made it from one sentence describing the scene, with no editing afterwards. Look at the details and it falls apart.

The same habit works here as with words: the overall impression is the part AI is good at, so check the small specifics. The fingers, shadows and reflections in this one hold up, which is why the text is where to look first. More on how the site's own AI pictures are made: How this picture was made.

Four habits that switch your doubt back on

1. Ask for sources, then open one. A link proves nothing until you have clicked it and found the claim on the page. If the tool can't search the web, ask it to point to the line in the material you gave it.
2. Check the numbers on their own. Every figure, date, price, name and deadline gets its own look, separately from the prose around it. Numbers are where a confident answer most often goes quietly wrong, and they are the parts people repeat.
3. Give it permission to say "I don't know". Tell it that an honest gap is better than a guess, and ask it to mark what it found in your material and what it filled in from general knowledge.
4. Set your bar by the stakes. A brainstorm needs no checking. A draft for your own eyes needs a quick read. Anything you send, sign, publish or act on with money needs the full check, every time.

Before you answer, follow these rules:
- For each claim, say where it comes from: [the document I gave you / a source you can link / your general knowledge].
- If something isn't in what I gave you, say so plainly instead of filling the gap.
- Mark anything you are unsure about with "Unsure:" and say what would settle it.
- List every number, date and name in your answer at the end, so I can check them one by one.
- "I don't know" is an acceptable answer.

My question: [your question]

Common mistakes

• Asking "are you sure?" in the same chat. It will often apologise and change a right answer, or repeat a wrong one with more conviction. Check against a source instead.
• Counting a link as a check. Some links don't exist, and some don't say what the answer claims. Open it.
• Checking the hard parts and skipping the boring ones. The wrong date in the second paragraph does more damage than the clever argument.
• Treating a hedge as a failure. "I'm not sure" is the most useful sentence an AI can give you on a question that matters.

Goes with the $25 webinar on Tuesday, November 10, 2026: When AI is wrong, and how to tell. Its reading list is in the Library.