Wiki · Concept · Last reviewed June 16, 2026

Conformal Prediction

Conformal prediction is a method for turning a model's output into a prediction set or interval with a chosen long-run coverage rate, using calibration examples rather than trust in the model's internal confidence.

Definition

Conformal prediction is an uncertainty-quantification method that wraps around a predictive model and returns a set of plausible answers rather than only one answer. In classification, the set may contain several labels. In regression, it may be a numeric prediction interval. If the user asks for 90 percent coverage, the procedure is designed so that the true answer falls inside the produced set about 90 percent of the time across future comparable cases.

The guarantee is statistical. It applies over repeated examples under assumptions such as exchangeability, often approximated by independent and identically distributed calibration and test data. It does not mean that one case is 90 percent safe, that every subgroup receives equal protection, or that a weak model becomes strong. A poor model can still be wrapped conformally, but its sets may become too large to use.

Conformal prediction belongs near Confidence Calibration, AI Evaluations, AI Hallucinations, and Human Oversight in AI. Its practical question is: what set of answers can the institution defend at this risk level?

How It Works

A common split-conformal workflow gives data three roles. Training data fit the underlying model. Calibration data, held out from training, measure how far predictions tend to be from the truth. New inputs then receive sets or intervals widened according to a quantile of those calibration errors.

The key object is a nonconformity score: a measure of how unusual or wrong a candidate answer looks compared with calibration examples. For regression, a simple score can be an absolute residual. For classification, it may use predicted probabilities or other label scores. The method includes candidates whose scores are conforming enough to meet the chosen error rate.

Original conformal prediction was developed for online prediction, where labels are revealed over time. Modern split conformal prediction is easier to attach to existing systems. Conformalized quantile regression combines quantile regression with conformal calibration so intervals can adapt to inputs with different variability while retaining finite-sample coverage guarantees.

Current Context

Conformal prediction has become more visible because many deployed AI systems are useful but not transparent enough to justify blind reliance. Angelopoulos and Bates describe it as a practical way to create uncertainty sets or intervals for black-box models in high-risk settings. The method is attractive because it can be applied after training and can work without assuming a specific probability model for the data-generating process.

For foundation models, the fit is uneven. Conformal methods are most direct when there is a well-defined target and scoring rule: classification, extraction, ranking, forecasting, risk stratification, multiple-choice answering, or structured outputs. Open-ended text generation is harder because "the true answer" may be plural, subjective, or expensive to label. A conformal wrapper around an LLM answer therefore needs a defined task, scoring function, calibration distribution, and failure policy.

Regulators and standards bodies rarely require conformal prediction by name, but they increasingly ask for the kind of evidence it can support. NIST's AI Risk Management Framework names valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair systems as trustworthiness characteristics. The EU AI Act requires high-risk AI systems to maintain appropriate accuracy, robustness, and cybersecurity through the lifecycle, and to support human oversight that can interpret and override system output.

Governance and Safety

The governance value of conformal prediction is that it makes uncertainty operational. A system can answer only when the prediction set is small enough, escalate when the interval is too wide, or ask for more evidence when coverage would otherwise be purchased by vagueness. That links model behavior to institutional rules: answer, abstain, retrieve, review, defer, or reject.

The safety limits are just as important. Marginal coverage can hide subgroup failures. A 90 percent guarantee over the whole population may still under-cover a language group, hospital, region, document type, or edge case. Calibration data can go stale when deployment data shift. Tools, retrieval, memory, or agent loops may change the effective task after the wrapper was tested.

Conformal prediction also does not solve misuse by presentation. A wide interval can be minimized, hidden behind a single recommendation, or translated into a false yes-or-no decision. Human oversight requires interfaces that show when uncertainty is driving abstention or escalation.

Defense Pattern

Spiralist Reading

Conformal prediction is a discipline for refusing fake precision.

The machine wants to compress uncertainty into an answer. The institution wants an answer it can route, sell, deny, approve, or automate. Conformal prediction interrupts that compression by returning a boundary: this much is covered by the calibration record, and beyond it the system must slow down.

Open Questions

Sources

Return to Wiki