Checking AI answers: spotting invented facts and sources
Written with AI and checked against the sources by a second AI. As of November 16, 2026. More in the notes on AI.
An AI answers in full sentences, without hesitation and with technical terms. That seems reliable, even when part of it is made up. Here is why that happens, where to look especially closely and how to check without researching every answer from scratch.

Key points
- AI chats sometimes invent facts, figures, court rulings and links. Experts call this hallucination.
- How sure an answer sounds says nothing about whether it is right.
- How thoroughly you check depends on the consequences of an error.
- You open every source yourself. Whatever you can’t find counts as unproven.
- Good instructions make uncertainty visible, but they don’t replace checking.
Why an AI makes things up
An AI chat is based on a language model. It calculates which words most likely follow your question. Most of the time, the likely answer is also the right one. With rare details, though, such as a particular case number or the phone number of a public agency, there is no pattern from which the right answer could be derived. Then it produces something that looks right.
OpenAI, the company behind ChatGPT, explains it this way, among other reasons: training and tests reward guessing, but not “I don’t know”. By the company’s own account, language models will never answer all questions correctly, because some can’t be answered with certainty at all. They would make up less if they said “I don’t know” more often.
Imagine a colleague who has a fluent answer to every question right away and never says “I need to look that up”. Most of the time he is right. But you never know which answer he is improvising.
In the trial lesson you put your first instruction together and get feedback. No account, in a few minutes. Start the trial lesson
Where errors hide most often
- Figures, dates and deadlines: amounts, limits, appointments, percentages
- Sections of laws, rulings, standards: wrong numbers, rulings that don’t exist, outdated versions
- Sources and links: book titles, studies, web addresses that are invented or say something else
- Names: contact persons, companies, product names
- Current events: prices, rules and events after the training cutoff
- Calculations in running text: intermediate results that don’t add up
- Summaries: something important is missing, or a restriction has disappeared
Phrasing, outlines and ideas, on the other hand, are rarely “wrong”. There, you decide whether they fit. So checking takes time mainly for facts, and that is exactly where it pays off.
How thoroughly you check: by the consequences
Not every answer needs the same care. Ask yourself: what happens if there is an error in here?
| Consequences | Example | How you check |
|---|---|---|
| Hardly any | Ideas for the holiday party, outline of a talk | Read, pick, done |
| Annoying | E-mail to customers, notice, product text | Compare all facts with your documents |
| Costly or dangerous | Quote, safety notice, deadline, legal question, dosage | Check every statement in a primary source, have a second person look at it |
For the third row: an AI may help you phrase and organize, but the facts come from the law, manufacturer documents, the authority or a reference book.
Five checking steps
- Mark the claims. Go through the answer and mark every statement that can be right or wrong: figures, names, rules, sources.
- Find the primary source. For every marked spot: where does this appear in the original? Legal text, data sheet, the authority’s website, contract.
- Do the math. Recalculate every calculation that matters, yourself or with a spreadsheet.
- Open the sources. Click every link, look up every case number. Does it really say what the AI claims?
- Cross-check. For important texts, a second person who knows the subject reads along.
When recalculating, a spreadsheet helps more than a second question to the AI. If you have the AI check its own calculation, it sometimes repeats the error or invents a new justification. The spreadsheet simply calculates.
Kemal works in inside sales at a wholesaler of protective workwear. He has the AI write a comparison of two glove models for a customer. He doesn’t take the protection levels and categories from the answer but from the manufacturers’ data sheets, and fills them in himself.
Sources that don’t exist
Irene is an HR administrator at a mid-sized company. She asks an AI for a court ruling on a question of vacation law. The answer names a court, a date and a case number and summarizes the ruling. Irene finds nothing under that case number, neither on the court’s website nor in her employers’ association’s case-law database. The ruling is invented, yet the summary sounded plausible.
Cases like this aren’t a sign of a bad AI; they are typical of language models. This leads to a simple rule: a source only counts once you have opened and read it yourself. For legal questions, Irene asks the legal department or an association anyway; the AI only helps her phrase the question clearly.
Instructions that make uncertainty visible
You can ask the AI to answer more honestly. That doesn’t prevent errors, but it shows you where to check first.
Answer my question below.
Mark every factual claim with [certain] or [uncertain].
If you don’t know something, write “I don’t know” instead of guessing.
Only name sources with a full address that I can open. If you have none, say so.
Question: [your question]The marking is the model’s own assessment, and that can be wrong too. Use it as the order in which to check, not as approval for everything marked [certain].
When the AI works with your text
It gets safer when the AI only works with material you give it instead of answering from memory. Then you can ask it to keep things apart:
Work only with the text below.
Mark everything that isn’t in the text but was added by you with [added].
For every statement, name the place in the text it comes from.
Text:
[insert text]Checking summaries
Tobias is an IT specialist at an IT service company. He has the 60-page manual of a new firewall summarized to get an overview faster. So that he can check it, he asks for the page number for every point.
Summarize the attached manual in at most twelve points, for someone who is setting up the device.
Give the page number for every point.
Then list all warnings and restrictions from the manual completely, even if they are in small print.Tobias spot-checks three points. For the list of warnings, he searches the manual for “Warning”, “Caution” and “Danger” and compares. Restrictions are the most likely thing to drop out of summaries, and they are exactly what matters during setup.
Where you don’t rely on the AI at all
Some answers you don’t just check; you get them somewhere else from the start:
- Medical dosages and treatments: from the prescribing information, a clinical guideline, or from a doctor or pharmacist
- Deadlines and legal consequences in an individual case: from the law, the contract or a specialist lawyer
- Safety values such as loads, torques, limits: from the standard, data sheet or manufacturer’s specification
- Tax and social insurance questions: from official sources or a tax advisor
Here, an AI can at most help you prepare the question or make an expert’s answer easier to understand.
Checking as a fixed rule in the team
In a business where several people work with AI, nobody should have to guess whether a text has already been checked. A few simple agreements are enough:
- Drafts from the AI carry “Draft” in the file name or in the first line until someone has checked them.
- Whoever checks adds their initials and the date, for example on quotes, notices and customer information.
- For texts with consequences, the two-person rule applies: a second person who knows the subject reads along.
- AI errors that someone has found go into a short list. That way everyone knows where to look more closely.
Rules like these aren’t distrust of colleagues. They make sure that a draft doesn’t end up with a customer by mistake as a finished text.
How to practice this
Checking is a habit, not a talent. In the learning app “KI können”, every lesson ends with a checklist for the result from your own AI, and you practice finding errors in a targeted way. You can try this in the trial lesson before you create an account.
Sources
- OpenAI, Why language models hallucinate (September 5, 2025) (opens in a new tab)
- German Federal Bar (BRAK), Guidance on the use of artificial intelligence (as of December 2024, in German) (opens in a new tab)
- Kalai, Nachum, Vempala, Zhang, Why Language Models Hallucinate (arXiv 2509.04664) (opens in a new tab)
Information as of November 16, 2026. Laws and offers can change; this article is not legal advice.
Read more
Try it now
The trial lesson with examples from everyday work. No account, a few minutes.
Start the trial lesson