The Hop Count Becomes the Clinical Risk Score
The June 2026 arXiv paper Compositional Reasoning Depth Predicts Clinical AI Failure, by Sanjay Basu, studies clinician-generated electronic-health-record questions and argues that the number of reasoning steps a question requires can predict where clinical AI systems fail.
Aggregate Accuracy Hides the Cliff
The paper, arXiv:2606.16890 [cs.CL], was submitted on June 15, 2026. Basu's object is not a bedside product launch or a claim that a model can replace clinical judgment. It is a measurement problem: if a benchmark mixes simple lookup questions with questions that require several linked inferences, the average score can hide a predictable failure gradient.
The study annotates 313 clinician-generated MedAlign EHR question-answer pairs across four hop levels and evaluates 301 of those questions. A hop is a distinct reasoning step required to answer a clinical question from the electronic health record. The paper reports that Claude Sonnet 4.6, GPT-4o, and an OpenAI model identified as gpt-5.4-2026-03-05 each show lower accuracy as hop count rises.
That result matters because "clinical AI accuracy" is too coarse a phrase for deployment. A system that handles one-hop retrieval tolerably may still fail when the question requires integrating a medication, a lab trend, a time relation, and a contraindication. The user experiences both as one answer box. The safety case should not.
What Hop Count Measures
Hop count is not a diagnosis, a severity score, or a full model-safety metric. It is a small annotation placed on the question: how many separate inferential moves are needed before an answer can be justified from the record? That makes it useful precisely because it is cheap enough to attach to ordinary evaluation sets.
A hospital, vendor, or auditor does not need to know the internal mechanism of a model to ask whether performance decays with question depth. The annotation belongs to the task. If the model's reported accuracy is high overall but collapses on high-hop questions, the institution has learned something operational: it should route those questions differently, require stronger retrieval evidence, or forbid automated answers in that class.
This page is distinct from the site's existing entries on black-box health LLM evaluation, pathology second readers, and patient-portal reply systems. Those pages focus on evaluation access, image-diagnostic workflow, and clinical voice. Basu's paper isolates a simpler lever: the structure of the question can be a risk feature before any answer is generated.
Why Thinking Longer Did Not Fix It
The paper also tests whether extended thinking flattens the accuracy-depth curve. According to the arXiv abstract, it does not significantly do so across the tested reasoning conditions, even though thinking-token usage scales with hop count. The model spends more compute as the questions become deeper, but the extra compute does not erase the decline.
That is the wrong shape for a deployment policy based on assumption. "Let the model think longer" is attractive because it feels like a local fix: same interface, same user flow, more hidden work. Basu's result suggests that for EHR question answering, longer test-time reasoning should be treated as an intervention to measure, not as a safety guarantee.
The context-sufficiency audit is important here. The paper reports that higher-hop questions were not merely more likely to be impossible because the relevant EHR evidence had been truncated. That does not prove every record is complete or every evaluation design is perfect. It does weaken the easy excuse that the slope is only a context-window artifact.
Clinical Governance Lesson
The practical lesson is not that every clinical AI system is unsafe in the same way. It is that aggregate accuracy is an unsafe governance surface. A deployment review that reports one overall score, or one score per specialty, can miss the failure mode the clinician actually faces: the hard cases are hard because they require composition.
Clinical governance should preserve this distinction. A question that asks for a single documented fact is not the same as a question that asks whether several facts jointly imply a medication risk, a discharge concern, or a need for escalation. The interface may flatten them into one chat turn. The review process has to unflatten them.
Hop count should therefore become a routing variable. Low-hop answers might be eligible for drafting, summarization, or clerical support after validation. Higher-hop answers should require stronger evidence display, clinician confirmation, or hard refusal depending on the risk of action. The point is not to crown a new metric. The point is to stop pretending that all questions with the same top-line accuracy impose the same clinical risk.
What It Does Not Prove
The paper does not prove that a named hospital, vendor, or production clinical assistant is unsafe. It studies a benchmark task, reports model behavior under specified conditions, and presents hop count as a theory-motivated predictor of error. That is evidence for evaluation design, not a product-specific incident report.
It also does not prove that hop count is the only risk variable. Clinical topic, patient population, record completeness, retrieval method, judge design, answer format, confidence calibration, and downstream action all matter. A one-hop answer can still be dangerous if it is about the wrong patient, the wrong time, or an irreversible decision.
Finally, the paper should not be read as saying that language models cannot help with clinical work. It says the work must be divided more honestly. Some tasks are clerical, some are summarizing, some are evidentiary, and some are multi-step clinical reasoning. Governance fails when one benchmark number lets those categories blur.
Governance Standard
A clinical AI evaluation card should report accuracy by hop count, not only in aggregate. It should also list question source, annotation protocol, model version, prompt and tool configuration, EHR context limits, retrieval method, answer-judging method, confidence calibration, and what happens when the model cannot justify an answer from the record.
For deployed systems, the routing rule should be visible. Which hop classes can be drafted for review? Which require evidence panels? Which are automatically escalated to a clinician? Which are out of scope? A score that does not change workflow is just decoration.
The Spiralist rule is this: the clinical question is part of the safety case. If the question requires more linked reasoning than the system can reliably perform, then the right answer is not a smoother paragraph. It is a boundary.
Sources
- Sanjay Basu, Compositional Reasoning Depth Predicts Clinical AI Failure: Empirical Evidence Consistent with Transformer Compositionality Limits in Electronic Health Record Question Answering, arXiv:2606.16890 [cs.CL], submitted June 15, 2026.
- arXiv experimental HTML for Compositional Reasoning Depth Predicts Clinical AI Failure, reviewed June 25, 2026.
- Related pages: AI in Healthcare, The Health LLM Becomes the Black-Box Examiner, The Pathology Model Becomes the Second Reader, The Patient Portal Reply Becomes the Clinical Voice, and The Sepsis Alert Becomes the Triage Bell.