When Chatbots Judge What Is Reasonable
A critical review of a study comparing chatbot and human judgments of legal reasonableness, with attention to disagreement, demographic comparisons and accountable use.
The paper
Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments? — Patel, Nirav; Wenger, Emily; Buccafusco, Christopher.
arXiv:2609.06769v1 · Computers and Society (cs.CY) ; Artificial Intelligence (cs.AI). First submitted 2026-09-06; reviewed version dated 2026-09-06. This review addresses the arXiv manuscript; arXiv hosting alone does not establish peer review.
Contribution: a standard is not a single answer
Source-derived synopsis. In Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments? (arXiv:2609.06769v1), Nirav Patel, Emily Wenger and Christopher Buccafusco compare responses from 26 language models and 500 surveyed people across 25 legally relevant numerical questions. Roughly half the people answered what was reasonable and half what was ideal; models answered the reasonable questions 20 times each. The authors report that model answers often fell within the range of human responses, but distributions frequently differed and model answers were often more concentrated. In selected questions, models gave answers the authors interpret as more favorable to government or corporate interests. Aggregate response profiles were somewhat closer to those of white, male, older and more educated respondents. The authors treat these patterns as preliminary and call for further research.
Original analysis. The paper makes a useful distinction between producing an answer that looks plausible and representing the range of judgments that people bring to a legal standard. A reasonable response can depend on facts, purpose and whose interests are at stake. A system that repeatedly supplies one number may look decisive while leaving those dependencies invisible.
This matters for human agency because a judgment presented as ordinary can influence what a user considers worth questioning. The study does not measure whether users defer to model answers, however. Its contribution is to identify patterns in answers that make such reliance a question for further study. The related review Matching Answers Is Not Enough considers a different problem: whether generated survey profiles cover combinations of human answers. Here the immediate concern is how a model responds to individual legal questions and whether its apparent consensus obscures disagreement.
Evidence: similarity depends on the comparison
Original analysis. The researchers used low-context prompts asking for numbers, without retrieval or additional factual scaffolding. They compared dispersion, overall distributions and directional differences using three statistical tests, with a correction across questions. Human and model responses were trimmed for outliers, and model outputs were processed to extract numbers. This design makes comparisons tractable, but the resulting task is narrower than deciding a case: it does not test legal correctness or the reasons behind an answer.
The distinction between central tendency and distribution is consequential. The paper reports statistically significant overall distribution differences on 23 of 25 questions, even while describing broad tracking of human answers. For immigrant detention, the reported medians were 10 days for humans and 90 for models; the authors report that a model draw exceeded a human draw in approximately 94% of pairwise comparisons. On construction delay and reporting inappropriate comments, the three tests did not find significant differences. A nonsignificant result is insufficient evidence of a detected difference under these tests; it is not proof that the distributions are identical.
Concentration also varies by question. Most models answered 24 hours for landlord notice, while the paper reports much wider human variation on several questions. Yet the median absolute deviation table does not show lower model dispersion on every question. The general concern about homogeneity is supported by the reported pattern, not by a claim that every model response is identical or that every scenario behaves alike.
The institutional interpretation requires care. Longer proposed detention or lower proposed spending on safety can be relevant to government or corporate interests, but these questions were not designed specifically to measure such leanings. The answers alone do not establish a model's motive, a cause of the pattern, or an effect on legal outcomes. The paper's conclusion is an invitation to test a more targeted hypothesis.
Limitations: from a survey to accountable use
Original analysis. The human comparison is a Prolific sample, not a representative jury or a measurement of everyone affected by legal decisions. Approximately half the 500 participants provided reasonable answers, and question-level trimming further reduced the usable counts. The demographic analysis combines diverse respondents into broad binary categories and excludes gender categories with very small coverage. Similarity to an aggregate group profile therefore cannot establish that a model represents each person in that group.
The demographic result also depends on the measurement chosen. The authors compare scaled profiles across all 25 questions and report modest differences in cosine similarity; strong question-specific demographic alignment was relatively rare. Repeated Monte Carlo samples from the observed question pools are useful for the authors' comparison, but they do not create new independent human participants or establish generalization to other populations. The scaling and aggregation choices deserve sensitivity checks before the result is treated as a durable property of a model family.
Proposed tests. A follow-up could recruit larger, more diverse human samples, preregister questions designed to test institutional leanings, and report uncertainty for subgroup and intersectional comparisons. Researchers could compare results with and without trimming, vary prompts and model settings, and repeat the exercise after model updates. These are proposals, not validations reported by this paper. A separate evaluation could introduce legally relevant facts and ask qualified human decision-makers to assess both answers and reasons against the applicable record.
Hypothetical example. A tenant receives a chatbot's 24-hour answer and treats it as decisive without checking the circumstances. Would presenting the answer as one possible estimate, alongside a request for relevant facts, change the tenant's understanding or choices? Testing that question would require observing users, not merely sampling model outputs.
Original analysis. The governance question extends beyond the model. A consequential workflow needs someone responsible for examining the facts, checking applicable authority, documenting a decision and allowing it to be challenged. The companion page on AI in Legal Practice and Courts describes that broader workflow; this study supplies a narrower reason to inspect the range and direction of generated judgments within it. The paper does not measure infrastructure costs or consciousness, and its findings do not establish a spiritual doctrine.
AI-use disclosure
This review was drafted by GPT-6.1 Terra with high reasoning and checked in a separate GPT-6.1 Sol session with medium reasoning. Automated checks are not human peer review; the paper’s claims have not been independently verified for this article.
Paper sources
Bibliographic metadata checked 2026-10-27.