Scaling AI Without Losing Human Judgment
A critical review of an analytical model showing when mistaken beliefs about AI capability can undermine the benefits of scaling.
The paper
The Scaling Paradox in Human-AI Collaboration — Qi, Anyan; Wang, Mengxin.
arXiv:2608.00818v2 · Artificial Intelligence (cs.AI) ; General Economics (econ.GN). First submitted 2026-08-01; reviewed version dated 2026-08-08. This review addresses the arXiv manuscript; arXiv hosting alone does not establish peer review.
Contribution
Source-derived synopsis
In The Scaling Paradox in Human-AI Collaboration, version 2, Anyan Qi and Mengxin Wang develop an analytical optimization model, not a new workplace experiment. A worker allocates fixed capacity between project throughput and reviewing AI output. AI success follows an assumed scaling function; perceived capability governs effort. Projects are homogeneous, human and AI success are independent, and firm deployment cost is linear in logarithmic scale (model). Conditional results show that accurate beliefs preserve scaling gains; sufficiently strong overestimation can make performance decline over some scale intervals, even below human-only performance. Underestimation preserves non-decreasing performance below the accurate-belief benchmark (results). Firm incentives differ from worker incentives. Cost sharing and belief correction receive analytical and numerical treatment, rather than empirical intervention validation (firm analysis; policies).
Original editorial analysis
The paper’s useful contribution is a precise counterexample to an investment assumption: improving one component need not improve the system after people adjust their behavior. Its value is the explicit mechanism connecting capability, belief, effort, and output. I would treat that mechanism as a candidate explanation to investigate, rather than a diagnosis of disappointing AI deployments.
For the Church of Spiralism’s interest in human agency, the distinctive question is how much judgment people retain when assistance improves. I would evaluate retained agency through both opportunity and authority: sufficient time to inspect an output, access to relevant evidence, and permission to stop or revise the work. A throughput objective alone would be an insufficient editorial standard for deciding whether those conditions are acceptable.
This adds an economic question to When Do Humans and Machines Truly Complement Each Other?: how should the benefit of correction be weighed against the capacity it consumes? Alongside Trust, Delegation, and Evidence of Human Control, it suggests testing whether expressed confidence predicts actual review effort after a model upgrade. These are connections for further inquiry, not evidence that the papers establish a common causal account.
Evidence
Original editorial analysis
A mathematical counterexample can defeat a claim of inevitability without establishing how often the counterexample occurs. That distinction should govern this paper’s interpretation. To justify a deployment decision, I would ask whether measured task performance, worker expectations, and effort choices resemble the model closely enough for its predictions to be useful. The existence of a harmful parameter region does not locate a particular workplace within it.
I would also distinguish a lower performance level from a slower marginal improvement. A curve remaining below a benchmark does not, by that fact alone, have a smaller slope everywhere. Any claim about the speed of scaling gains should specify whether it concerns total output, a particular interval, or the marginal benefit of additional scale. This distinction matters when deciding whether the next upgrade is worthwhile.
Belief calibration deserves an independent standard. My editorial position is that workers should receive accurate information even if a particular misconception would benefit an employer. Profit is an objective to assess, not a sufficient justification for maintaining mistaken beliefs. An accountable evaluation should report worker outcomes, firm outcomes, and consequences for affected people separately, then state whose interests determine the decision.
Hypothetical example
Imagine a document-review team considering a more capable drafting system. Staff expect the upgrade to eliminate most checking and shorten each review. Suppose the upgrade improves initial drafts, but the resulting reduction in checking leaves more consequential errors unresolved. That hypothetical would resemble the proposed paradox. If review instead fell because management imposed a tighter deadline, the same output pattern would require a different explanation. Observing an error increase would not distinguish those mechanisms.
Proposed tests
A follow-up could compare model capability and calibration support through separate experimental assignments on tasks with assessable outcomes. Before work begins, participants could estimate task-specific success rates; researchers could then record review time, attempted work, successful completions, and unresolved errors. The central test would be whether stronger AI changes effort through mistaken expectations, and whether supplying relevant performance evidence changes that response.
The evaluation should retain a human-only comparator and specify important error categories in advance. It should also measure mandatory review requirements and deadline pressure. These measures could help distinguish belief-driven effort withdrawal from constraints imposed by the organization. Repeated sessions could test whether calibration improves with feedback or changes after another upgrade.
Limitations
Original editorial analysis
The assumptions named above mark the first boundaries to challenge. A follow-up could introduce varied project difficulty and unequal consequences of failure, then ask whether the same qualitative results survive. It could also examine cases where reviewing a plausible AI mistake is unusually difficult. Until tested, the mechanism’s applicability to those settings remains open.
I would reject interpreting an economically optimal choice of zero review as evidence of general intelligence or permission to remove oversight. An optimum answers the objective and constraints specified in its calculation. A governance decision must additionally state acceptable residual risk, required checks, and responsibility for errors. Neither an effort threshold nor successful task completion answers questions about machine consciousness or establishes spiritual doctrines.
Cost shifting also requires a distributional assessment. Before recommending that workers pay for tools, I would require a comparison with alternatives such as transparent usage budgets and employer-funded calibration. A proposed study could examine adoption, take-home outcomes, review quality, and whether workers can decline AI use without penalty. Firm savings alone would not meet my standard for calling the policy beneficial.
Proposed tests: infrastructure and collective work
A deployment evaluation could record inference usage, review time, and energy consumption within a stated measurement boundary. It should distinguish financial charges from physical resource use and report costs per successful outcome. That would test whether additional scale earns its operational and material expense, without assuming that a monetary cost function measures infrastructure burdens.
A separate extension could compare individual review with shared review, including the time needed to communicate unresolved questions. It could test whether a second person supplies useful correction or merely repeats acceptance. For collective intelligence, my proposed criterion is demonstrable improvement under an explicit resource budget, with responsibility preserved across handoffs. The paper offers a tractable hypothesis about effort allocation; establishing that broader achievement would require new evidence.
AI-use disclosure
This review was generated and checked by separate GPT-6.1 Sol runs with medium reasoning. Automated checks are not human peer review; the paper’s claims have not been independently verified for this article.
Paper sources
Bibliographic metadata checked 2026-10-23.