Catch the made-up quote

Ask an AI to summarise a report, an article or a transcript and it will often give you quotes. Some of them will be real. Some will be close but tidied up, a word changed here, a sentence merged there. A few will be invented outright, in quotation marks, sounding exactly like the speaker. Quotation marks are a promise that those are the exact words, so this is the one place where close isn't good enough.

The good news: this is the easiest AI error to catch. It takes about a minute and needs nothing but the original and your find key.

The find test

1. Open the original: the report, web page, PDF or transcript, not the AI's summary of it.
2. Press Ctrl+F (Cmd+F on a Mac) and search for five to eight distinctive words from the quote. A short chunk works better than the whole quote, because punctuation and line breaks can stop an exact match.
3. No match? Try a different chunk from the same quote. Then try one key word on its own and read what's there.
4. Found it? Read the sentence before and after. A real quote can still mislead if the next line says "but that's not what we found".
5. Not found? Mark it NOT FOUND and don't use it. Don't repair it yourself and don't ask the AI to fix it: a repaired quote is still one nobody said.
6. Use what the source actually says, in its exact words or as a clearly marked paraphrase.

For each quote you gave me above, show the full sentence before it and the full sentence after it, copied from the source exactly as written.
If you cannot find a quote word for word in the source, write NOT FOUND for that quote. Do not repair, reconstruct or paraphrase it.

This prompt helps, but it is not the check. The AI can be wrong about its own quotes too, so still run the find test on anything you'll use.

Quotes from a recording

• Go to the timestamp and listen to about 20 seconds around it. Transcripts get names, numbers and jargon wrong more than anything else, and those are the bits you'll want to quote.
• If you have two transcripts (the meeting tool's own and one you made), compare them. Where they disagree on a name or number, the audio decides.
• Watch for a sentence repeated many times in a row. That is the transcriber going wrong, not the speaker.

Here are two transcripts of the same recording. A: [where it came from]. B: [where it came from].
List every place where they differ on a name, a number, a date or a technical term, with the timestamp. Don't decide which is right. I will check the audio.

A:
[paste]

B:
[paste]

Common mistakes

• Asking the AI "is this quote accurate?" It will often say yes.
• Fixing a quote so it matches the source. If it wasn't there, there's nothing to fix.
• Trusting a quote because it sounds like the person. Invented quotes are built to sound right.
• Checking the words and not the context. Read the lines around it.