What are hallucinations in LLMs and why they happen

A hallucination in a language model (LLM) is a response that sounds plausible and confident but is factually incorrect or not backed by any source. The model isn’t lying —lying requires intent— rather, it fills a knowledge gap with the statistically most likely continuation, which is sometimes true and sometimes a convincing fabrication. The more precise technical term is confabulation: the system doesn’t know that it doesn’t know.

To understand why it happens, it helps to start with a question that seems simple and isn’t.

Why does such an advanced system make things up?

The intuitive answer —“it lacks data”— is only part of it. The more interesting explanation appeared in 2025, in an OpenAI paper titled Why Language Models Hallucinate, and it uses an analogy worth keeping in mind: a school exam.^[1]

Imagine a multiple-choice exam where leaving an answer blank scores zero points, but guessing has a chance of being right. What does a rational student do? Always guess. Never leave anything blank, because guessing is never worse than abstaining. According to the OpenAI paper, language models permanently live in that exam: they are trained and evaluated with binary metrics (right / wrong) that penalize “I don’t know” exactly like a wrong answer. The result is that the system learns that guessing confidently is the winning strategy, because admitting uncertainty only costs it points.^[1:1]

This is the twist that changes how to understand the problem: hallucination is not just a technical flaw of the model, it’s partly an incentive problem created by the way we measure AI. A model that said “I’m not sure” more often would be more reliable, but scores worse on benchmarks that reward always answering.

To that underlying cause are added others, more familiar:

• Gaps in the training data. On rare, specialized or post-cutoff topics, the model has no signal to draw on. It answers anyway, and does so with confidence.^[2]
• Ambiguous prompts. A vague instruction gives the model more latitude to drift toward coherence instead of accuracy: it optimizes for sounding good, not for being right.^[2:1]
• Poor-quality context. Even with information retrieval techniques (RAG), if what the system retrieves is poor or outdated, the answer will come out wrong. RAG is only as good as what it brings in.^[2:2]

How often does it happen? (the uncomfortable part)

Often enough that no critical system can ignore it. The figures vary enormously depending on how it’s measured —and that variation is itself part of the message:

• The TruthfulQA benchmark reports hallucination rates above 50% in most base models.^[3]
• Stanford’s 2026 AI Index found rates of between 22% and 94% depending on the test, across 26 leading models.^[4]
• On tasks with well-done information retrieval (grounded), OpenAI’s evaluations show the rate can fall below 2%.^[3:1]

The conclusion is not “the models are bad”, but something more useful: the hallucination rate depends enormously on the use case, on the context you give it and on how you measure it. The same model can hallucinate 50% on open-ended questions and less than 2% when it answers anchored to reliable documents. That’s why you can’t rely on a model’s general reputation: you have to measure it in your application, with your data.

The types of hallucination you’ll encounter

Not all hallucinations are the same, and distinguishing them helps combat them:

The most cited real case belongs to the last category: in 2023, a lawyer submitted to a New York court an AI-generated brief that cited six court cases that didn’t exist, with convincing names and numbers. The judge sanctioned the lawyers.^[5] It was not an isolated event: between 2023 and 2025, hundreds of court decisions addressing hallucinations in legal briefs were documented.^[5:1]

Why this is a business problem, not just a technical one

In a personal experiment, a hallucination is a funny anecdote. In production, it’s liability. When a banking assistant invents an interest rate, an insurance agent misreads a coverage or a health chatbot suggests something incorrect, the cost is real —reputational, legal and financial. That’s why hallucinations matter especially in sectors where a wrong answer has consequences: banking, healthcare, legal and insurance.^[6]

And here there’s a dangerous asymmetry: the hallucination comes wrapped in confidence. The system doesn’t hesitate when it invents; it asserts it with the same confident tone it uses for what’s correct. As one analysis of the phenomenon summed it up, the danger is not that “the AI said something incorrect”, but that “it said it beautifully, and we believed it”.

How they are reduced (and why they aren’t eliminated completely)

The good news is that understanding of the problem has matured, and with it the tools to attack it. The most effective levers:

1. Anchor answers to reliable sources (RAG). It’s the most recommended mitigation and the highest-impact one: give the model curated documents to draw from instead of letting it “remember”. It drastically lowers hallucinations, but —importantly— it’s not a silver bullet: if retrieval fails, the system goes back to guessing.^[2:3][1:2]
2. Design the system so it can say “I don’t know”. A model configured to ask for clarification or abstain in the face of uncertainty is more reliable than one that always answers. This includes exposing confidence levels or citing sources.^[7]
3. Fidelity-oriented fine-tuning. Train the model to prefer answers faithful to the source; some studies report very large reductions in the hallucination rate with this technique.^[8]
4. Well-designed prompts and context. Narrowing the scope, giving a role and anchors reduces the latitude to wander —though, on its own, it doesn’t make up for poor data retrieval.^[2:4]

None of these techniques brings the rate to zero, and it’s wise to distrust anyone who promises otherwise. As long as models remain probabilistic and benchmarks keep rewarding guessing, a portion of hallucination is structural. What you can do —and what separates a reliable system from a dangerous one— is measure it, monitor it and keep it under control.

Measure before trusting

This is where theory becomes practice. Applying RAG and assuming the problem is solved isn’t enough: you have to verify, with real test cases, how often your agent invents, and watch it over time because the rate changes when the model or the context changes.

That’s what an evaluation platform like ArtificialQA does: it connects to your agent —by URL or API, without writing code— and subjects it to a hallucination evaluator that measures, over representative cases, when the answer departs from the truth or from the source. The result stops being a hunch (“I think the bot is doing fine”) and becomes a number you can track, compare across versions and show in an audit. Because with hallucinations the question is never whether they’ll happen, but how many you’re willing to tolerate —and that’s only known by measuring it.

---

Frequently asked questions

What is a hallucination in an LLM? It is a response that seems plausible and confident but is factually incorrect or not backed by any source. The model fills a gap with the most likely continuation, without knowing it’s false.

Why do AI models hallucinate? For a combination of causes: gaps in the training data, ambiguous prompts, poor-quality context and —according to OpenAI— an incentive problem, because benchmarks reward guessing confidently over admitting uncertainty.

Are hallucinations lies? No. Lying implies intent to deceive. The model has no intent: it just generates the statistically most likely sequence. The technical term is confabulation.

Can hallucinations be eliminated completely? Not entirely. As long as models are probabilistic, a portion is structural. They can be greatly reduced with RAG, fine-tuning and good prompt design, and kept under control by measuring and monitoring them.

Does RAG solve hallucinations? It reduces them significantly by anchoring answers to reliable sources, but it’s not a complete solution: if retrieval brings in poor data or fails, the model goes back to guessing.

How is a chatbot’s hallucination rate measured? By subjecting it to representative test cases with an evaluator that detects when the answer departs from the truth or the source, and monitoring that rate over time, since it changes when the model or the context changes.