Confidence and Judgment in Human–AI Reasoning
A critical review of an exploratory study separating AI-assisted reasoning performance from people’s ability to assess their own answers.
The paper
Confident, Not Wiser: The Dunning-Kruger Effect in Human-AI Interaction — Fernandes, Daniela; Rausch, Michelle; Kloft, Agnes Mercedes; Buschek, Daniel; Welsch, Robin.
arXiv:2609.31095v1 · Human-Computer Interaction (cs.HC). First submitted 2026-09-25; reviewed version dated 2026-09-25. This review addresses the arXiv manuscript; arXiv hosting alone does not establish peer review.
Contribution
Fernandes and colleagues examine whether people who perform reasoning tasks with AI can judge how well their joint work has gone. In an exploratory study of 366 UK adults, separately recruited groups completed 40 items alone or with a conversational model; assisted participants had to message it at least once per item. The authors also benchmarked the model on the same items. Participants estimated overall and block scores and rated confidence in each answer.
The assisted group scored 28.36 out of 40 on average, compared with 21.13 for the unaided group. Its difference from the model-only benchmark was uncertain. Both groups overestimated their totals by about two points on average. Confidence distinguished correct from incorrect answers less well in the assisted group across the full battery, but the smaller within-block difference was inconclusive. A performance-dependent self-estimation pattern remained in both groups after score-noise controls; the controlled difference between groups was uncertain. The authors fit an extended computational description and propose interface support for verification. Their versioned paper, arXiv:2609.31095v1, does not test those interface proposals.
Original analysis: The contribution is the separation of three questions that a single accuracy score cannot answer: did the work improve, did people know their overall result, and did their confidence help identify particular errors? For a community interested in human agency and collective intelligence, that separation gives oversight a measurable starting point. A person can participate in a decision while lacking a dependable signal for deciding which answer needs another look. The study measures aspects of that signal in a reasoning battery; it does not measure participants’ authority to challenge a consequential decision.
The related review Available but Unclaimed considers whether model competence becomes a benefit in assisted work. This paper adds a distinct question: whether people can recognize the quality of the work they produce together. The two questions should be assessed separately when deciding what a human–AI process is expected to accomplish.
Evidence and interpretation
Original analysis: The same-item model benchmark makes a useful comparison possible, but the comparison has a narrow meaning. The overall assisted mean exceeded the model benchmark by 0.65 points, with a 95% confidence interval from −0.27 to 1.57. That interval does not establish either overall synergy or equivalence. Matrix reasoning was the only block where the assisted group demonstrably exceeded both the unaided group and the model benchmark. The result is specific to that battery and benchmark protocol, rather than proof that a partnership will outperform both members on new work.
Average signed error also hides substantial individual differences. Mean absolute total-score error was about six points in both groups, even though each group’s signed mean error was about two. Overestimates and underestimates can offset one another in a group average. A report that presents only that average would give readers little sense of how far an individual estimate might be from the achieved score.
The confidence analysis raises a further distinction. Across all items, the mean discrimination measure was 0.649 unaided and 0.564 assisted, where 0.5 represents chance discrimination. Yet approximately 80% of the pooled correct–incorrect comparisons crossed task blocks. When comparisons were confined to the same block, the estimated group difference was −0.022, with an interval from −0.047 to 0.004. The narrower comparison leaves a meaningful uncertainty: it does not establish equal within-task discrimination. Perfect block scores also provide no correct–incorrect pairs, so selection at the score ceiling remains a concern.
The Confidence Calibration page discusses checking a system’s confidence against outcomes. Here the confidence being examined is the participant’s confidence in a joint answer. These are different objects: a well-calibrated model signal would still need to be understood and used appropriately by a person, and this study does not establish that displaying such a signal would improve judgment.
The Dunning–Kruger label also needs care. Ranking people on the same score subtracted from their estimate can create part of a low-versus-high performer contrast. The authors separated ranking and scoring across item halves; the contrast became smaller but remained positive in both groups. Their simulations challenge particular constant-bias and noise accounts, while the interval for the controlled group difference spans zero. This supports a residual pattern under the stated checks, not a demonstrated larger AI-induced deficit or a unique psychological explanation.
Limitations and proposed tests
Original analysis: Separate recruitment calls and the absence of random assignment limit causal conclusions about assistance. The study was not preregistered, used one model and four fixed task types, and required consultation without correctness feedback. The model-only benchmark was run after data collection, omitted instruction and practice context, and reset conversation history between blocks, unlike the live interaction. Repeated benchmark runs improve precision for these items but do not turn them into a representative sample of tasks. The paper reports one confirmed answer-extraction error and says other unflagged replies were not audited.
The extended computational model is a descriptive account, not an independently validated mechanism. Its comparisons with observed quartiles were in-sample, and it underpredicted some maximum-score estimates. The confidence decomposition additionally treats an Unsure–Certain slider as a probability scale; its relationship with score partly follows from subtracting that score. These limitations matter because an apparent pattern in reports can have several explanations without identifying how a person reasoned.
Hypothetical example: A worker uses an assistant to draft an eligibility assessment and feels certain of the final answer. Confidence alone would not show whether the worker checked the governing rule, followed a persuasive suggestion, or independently reached the same conclusion. A record of the evidence consulted and the reason for the decision would provide different information. The example illustrates an accountability question; it is not an observation from this study.
Proposed test: Randomly assign participants to optional consultation, mandatory consultation, and unaided conditions, then compare score accuracy, confidence discrimination, and estimates on fresh tasks. Record initial answers and revisions where feasible, and independently assess the correctness of both initial and final responses. A later unaided test could distinguish immediate joint performance from independent performance after assistance. These are proposed measurements, not established remedies.
Proposed test: Compare verification prompts or task-specific feedback with an ordinary interface, measuring corrected errors, unnecessary changes, time, and user control. Evaluate the prompts on new items rather than selecting them using the same answers used to judge success. Alongside any gain, report review labor, model calls, latency, expense, and measured energy use where available. The supplied study provides no estimate of those infrastructure costs.
Original analysis: Better reasoning scores do not by themselves establish that a system is aligned with an institution’s goals, that an affected person can contest its output, or that anyone has authority to stop a decision. Those require separate evidence and governance choices. Nor do these results establish anything about machine consciousness or spiritual doctrine. Their practical value is a more modest one: they show why evidence about performance and evidence about a person’s ability to assess that performance belong side by side.
AI-use disclosure
This review was drafted by GPT-6.1 Terra with high reasoning and checked in a separate GPT-6.1 Sol session with medium reasoning. Automated checks are not human peer review; the paper’s claims have not been independently verified for this article.
Paper sources
Bibliographic metadata checked 2026-10-27.