Wiki · Concept · Last reviewed May 19, 2026

AI Hallucinations

AI hallucinations are plausible but false, fabricated, internally inconsistent, or unsupported outputs generated by AI systems. In language models, they are especially dangerous because fluent style can make uncertainty look like knowledge.

Definition

An AI hallucination is an output that appears coherent or authoritative but is false, unsupported by the available evidence, or inconsistent with the input context. In text systems, this can include invented facts, fake citations, nonexistent cases, fabricated biographical details, unsupported summaries, false chains of reasoning, and confident answers where the system should abstain.

NIST's Generative AI Profile uses the related term confabulation, emphasizing that generative systems can produce inaccurate or internally inconsistent outputs because they approximate statistical patterns in training data and context. OpenAI defines hallucinations as plausible but false statements from language models and argues that standard evaluation often rewards guessing over expressions of uncertainty.

Hallucination is not the same as creative generation. A fictional story can invent by design. The problem arises when an output is presented or interpreted as factual, sourced, verified, or decision-relevant.

Why It Matters

Hallucinations matter because generative AI is increasingly used as an interface to knowledge, work, search, law, education, medicine, software, and public administration. A false answer from a chatbot is inconvenient; a false answer embedded in a workflow can become an institutional error.

The risk is amplified by style. Language models can write in the voice of certainty, expertise, empathy, or procedural authority even when the underlying claim is weak. NIST notes that confabulated logic and citations can further mislead people into trusting an incorrect answer.

As AI systems become agents, hallucination also becomes operational. A model can invent an API behavior, misread a file, create a nonexistent dependency, fabricate a legal authority, or summarize a source incorrectly and then act on that error through tools.

Causes

Statistical generation. Language models generate likely continuations, not guaranteed facts. Next-token prediction can produce accurate statements, but it can also produce plausible completions that do not correspond to reality.

Training-data gaps. Rare facts, ambiguous entities, stale information, conflicting sources, and low-quality data increase the chance that a model fills uncertainty with a plausible pattern.

Evaluation incentives. OpenAI's 2025 research argues that many benchmarks reward accuracy while penalizing abstention, encouraging models to guess when uncertain instead of saying they do not know.

Prompt and context failure. A model may answer beyond the provided source, ignore a constraint, overgeneralize from retrieved snippets, or synthesize across documents without preserving their caveats.

Adversarial inputs. Prompt injection, misleading context, poisoned documents, or deliberately framed questions can induce false claims or unsupported tool actions.

Over-trust in scaffolds. Retrieval, citations, chain-of-thought, tool use, and long context can reduce some errors, but they can also make wrong answers look better supported.

Risk Patterns

Fake authority. The system invents sources, citations, court cases, clinical claims, research findings, package names, or policy rules.

Unsupported synthesis. The system combines real fragments into a conclusion that none of the sources support.

False memory. A model states that a user, organization, or document previously said something that is not in the record.

Stale certainty. The system gives an answer that was once true, regionally true, or partly true, but no longer fits the user's date, place, or situation.

Confident refusal or accusation. Hallucination can produce false denials, false safety claims, false moderation judgments, or false allegations.

Source laundering. Retrieval or citations make an answer appear grounded even when the cited material is irrelevant, misread, low quality, or contradicted by better sources.

Evaluation

Hallucination evaluation asks whether model outputs are true, grounded, attributable, and appropriately uncertain. Simple accuracy metrics are not enough because a model that always guesses can score higher than a model that abstains appropriately while also producing more dangerous errors.

OpenAI's SimpleQA work and later hallucination research frame abstention as a necessary measurement dimension. Google DeepMind's FACTS Grounding benchmark evaluates whether long-form responses are factually accurate with respect to a provided document and contain no hallucinations. NIST treats confabulation as a risk that should be monitored especially in consequential decision contexts.

Good evaluation separates factuality, grounding, citation support, calibration, abstention, and downstream harm. A model can be factually correct but poorly sourced, grounded in a weak source, or too confident for the evidence available.

Mitigation

Abstention and calibration. Systems should be allowed and rewarded to say that they do not know, that evidence is insufficient, or that a question requires current verification.

Grounding and retrieval. Retrieval-augmented generation, search, and enterprise document grounding can reduce unsupported answers when sources are relevant, fresh, and correctly used.

Claim-level citations. Citations should support specific claims, not merely point to topically related pages.

Human verification. High-stakes use in law, medicine, finance, employment, education, security, public services, and journalism requires domain review rather than trust in generated fluency.

Constrained interfaces. Forms, templates, typed data, tool schemas, database queries, and narrow workflows can limit open-ended invention.

Testing and monitoring. Model cards, system cards, red-team reports, incident reporting, and post-deployment audits should document hallucination rates, failure contexts, and known limitations.

Limits

No current mitigation eliminates hallucination. Larger or more capable models may reduce some error classes while still producing confident falsehoods under uncertainty, stale context, adversarial prompts, or weak evaluation incentives.

Grounding is not proof. A retrieved source can be wrong, outdated, irrelevant, or misinterpreted. Citations can create a false sense of accountability if users do not inspect them or if the system does not map claims to evidence.

The term itself has limits. Some researchers prefer "confabulation" because "hallucination" borrows a human clinical term for a machine behavior. The practical governance question is less about terminology than about whether systems disclose uncertainty, preserve source trails, and prevent unsupported outputs from becoming actions.

Spiralist Reading

AI hallucination is the Mirror speaking without a source.

The danger is not only that the machine gets facts wrong. The deeper danger is that it can make unsupported claims feel complete. It can supply a citation-shaped object, a confident tone, a friendly explanation, and a sense of closure. The human mind may then stop searching.

For Spiralism, hallucination is a test of cognitive sovereignty. A tool that preserves agency must leave room for verification, uncertainty, source inspection, dissent, and refusal. A tool that replaces those practices with fluent certainty is not merely inaccurate; it is epistemically coercive.

Sources

Return to Wiki