Selective Prediction

Definition

A selective predictor combines a predictor with a selection rule. For an input x, the first proposes an output; the second accepts it or abstains. Coverage is the share accepted. Selective risk is the average loss on accepted cases. The risk-coverage trade-off asks how much coverage must be surrendered to reach a stated error level.

Selective prediction is an operating policy, not “knowing what it knows.” The selector may use a probability, margin, ensemble disagreement, retrieval result, or learned error detector. It works only if that score ranks likely failures in the deployment setting. A fluent “I am unsure” is not evidence unless a measured rule changes the action.

How It Works

1. Define loss and alternatives. Specify a wrong answer and the cost of abstention, delay, retrieval, or routing elsewhere.
2. Produce a selection score. Estimate whether the proposed output meets the task's acceptance rule. The score is distinct from the output.
3. Set an operating point. On separate calibration data, choose a threshold for target coverage, selective risk, or expected cost. Do not tune on the final test set.
4. Route both outcomes. Accepted outputs continue. Abstentions receive a reason and route: clarification, retrieval, another model, human review, or no decision.
5. Monitor the pair. Recheck risk and coverage after changes in data, model, prompt, tools, users, or fallback capacity.

Boundary Tests

• Not confidence calibration. Calibration compares scores with outcome frequencies; selection uses a score to choose cases. Useful ranking need not be a calibrated probability, and calibration does not supply the threshold or cost policy.
• Not conformal prediction. Conformal methods return sets, intervals, or risk controls under stated assumptions. Abstaining when a set is too wide is an additional policy.
• Not safety refusal. An epistemic abstention says evidence is inadequate. A policy refusal blocks disallowed assistance even when the model could produce it. Logs should preserve the distinction.
• Not deferral by itself. Abstention stops the model's answer. Deferral assigns another decision-maker and must account for competence, bias, workload, delay, and authority.

Current Context

Reject-option research reaches back to Chow's 1957 decision-theoretic work; El-Yaniv and Wiener named the modern risk-coverage framing in 2010. Later methods thresholded a trained network, as in Geifman and El-Yaniv's 2017 work, or trained prediction and selection together, as SelectiveNet did in 2019. These results do not make every confidence score safe for abstention.

Language systems make the selector harder to define. ACL 2020 experiments found raw softmax confidence weak under the tested question-answering domain shifts; a separate calibrator improved reported coverage at fixed accuracy. A 2024 Nature paper used semantic entropy to detect a defined subset of free-form errors called confabulations. Such detectors can supply a score, not identify every false, harmful, ambiguous, or unauthorized answer. As of August 24, 2026, selective prediction is best treated as an evaluation and routing framework, not a universal uncertainty detector.

Evaluation and Governance

A selective system should be evaluated at full coverage and across the intended operating range. Report risk-coverage or accuracy-coverage curves, the chosen threshold, uncertainty intervals, abstention rate, and the cost and outcome of the fallback route. High accepted-case accuracy without coverage is incomplete: a system can look excellent by answering almost nothing.

NIST's voluntary AI Risk Management Framework frames validity, reliability, and group impacts as contextual, lifecycle concerns; it does not certify an abstention threshold.

• Test shift. Re-evaluate on new domains, languages, input quality, adversarial cases, and after model drift; threshold validity does not travel automatically.
• Audit groups. ICLR 2021 experiments showed that average selective accuracy can improve while disparities widen, including across the five vision and NLP datasets tested. Report coverage and risk for relevant groups, not only an aggregate.
• Protect the fallback. A human-review queue is not an infinite safety resource. Measure delay, abandonment, rubber-stamping, reviewer disagreement, and whether abstentions silently become approvals.
• Separate reasons. Record uncertainty, missing evidence, ambiguity, out-of-scope input, tool failure, policy refusal, and capacity limits separately. One generic fallback label hides different remedies.
• Keep authority outside the score. A high score should not authorize an irreversible action. Permissions, consequence limits, and accountable human oversight remain separate controls.

Minimum Selective-Prediction Record

Record the system and task version; proposed output and ground-truth rule; score definition and provenance; calibration and test populations; threshold and selection date; full-coverage loss; selective risk, coverage, and uncertainty intervals; subgroup results; abstention reasons; fallback owner, capacity, cost, and outcome; user notice and recourse; monitoring window; incidents; and retest triggers. Link that record to AI Evaluations, AI Audit Trails, and AI Post-Market Monitoring.

Source Discipline

A claim should name the task, model, selector, calibration data, loss, target, achieved coverage, achieved risk, uncertainty interval, comparison baseline, subgroup analysis, and fallback. “The model knows when to abstain” is too broad. Evidence from closed-set classification does not establish reliable selection for open-ended text, tools, or changed deployment data.

Spiralist Reading

Selective prediction gives the machine a door marked “not this time.” The ethical work begins after it passes through. An abstention that reaches evidence, care, or accountable judgment can be restraint; one that merely exports delay and burden is risk displaced out of view.

Open Questions

• Who bears the cost when the system abstains, and is that burden measured?
• Which score remains reliable after retrieval, tools, or user interaction changes the task?
• When should a system clarify or gather evidence instead of immediately deferring?
• How should institutions cap automation while preserving timely human review?

Related Pages

• Confidence Calibration
• Conformal Prediction
• AI Evaluations
• AI Hallucinations
• Human Oversight of AI Systems
• Automation Bias
• Algorithmic Bias
• Model Drift
• AI Audit Trails
• AI Post-Market Monitoring

Sources

• C. K. Chow, An Optimum Character Recognition System Using Decision Functions, IRE Transactions on Electronic Computers, 1957.
• Ran El-Yaniv and Yair Wiener, On the Foundations of Noise-free Selective Classification, Journal of Machine Learning Research, 2010.
• Yonatan Geifman and Ran El-Yaniv, Selective Classification for Deep Neural Networks, NeurIPS 2017.
• Yonatan Geifman and Ran El-Yaniv, SelectiveNet: A Deep Neural Network with an Integrated Reject Option, ICML 2019.
• Hussein Mozannar and David Sontag, Consistent Estimators for Learning to Defer to an Expert, ICML 2020.
• Amita Kamath, Robin Jia, and Percy Liang, Selective Question Answering under Domain Shift, ACL 2020.
• Erik Jones, Shiori Sagawa, Pang Wei Koh, Ananya Kumar, and Percy Liang, Selective Classification Can Magnify Disparities Across Groups, ICLR 2021.
• Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal, Detecting Hallucinations in Large Language Models Using Semantic Entropy, Nature, 2024.
• NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 2023; reviewed August 24, 2026.

Return to Wiki