The Budget Router Becomes the Allocation Receipt
Teng-Ruei Chen's July 2026 arXiv paper formulates a practical inference question: when an LLM system has a fixed per-query budget, should it sample the same model again or reroute to another model?
An allocation receipt ties a model-routing claim to the model pool, cost vector, verifier, budget, latency, and step-by-step draw history that produced the answer.
The Paper
The paper is Teng-Ruei Chen's Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models, arXiv:2607.08665 [cs.LG]. The arXiv API lists version 1 as submitted on July 9, 2026, with code at github.com/luka-krixvon/resample-or-reroute-experiment and a companion analysis at arXiv:2607.03436. The PDF metadata reports 10 pages, and the arXiv HTML title page lists National Yang Ming Chiao Tung University and Krixvon affiliations.
The paper studies a narrow operational choice. A serving system may have several models, an imperfect verifier, and a fixed amount of money or latency it can spend. The question is not only "which model should answer?" It is "what is the next unit of inference budget buying?"
Why It Matters
Model routing is usually sold as a frugal layer: send easy queries to cheaper models and hard queries to stronger ones. Test-time sampling is usually sold as a reliability layer: draw multiple answers and choose the best. Chen makes those two moves compete inside one budget. If the system can afford one more call, it must decide whether to ask the same model again or try another.
That matters because routing is becoming ordinary infrastructure. A support bot, legal intake tool, coding agent, tutor, or public chatbot may hide the routing layer, while that layer decides which cases get repeated attempts, escalation, early stopping, or a merely adequate answer.
The Policy
The proposed resample-or-reroute policy, RoR, keeps a per-query estimate for each candidate model. That estimate combines an offline prior, calibrated on a training split, with verifier-scored draws observed during the current query. At each step, the policy scores another draw from a used model and a first draw from an unused model by estimated marginal correctness per unit cost.
The design choice is that resampling and rerouting use the same currency. A larger model is not automatically privileged, and a repeated draw is not automatically waste. The paper studies a greedy variant, a UCB variant with an exploration bonus, and a non-deployable oracle allocation as a ceiling.
The Experiment
The evaluation replays regenerated multi-draw correctness tensors rather than running live inference during every policy comparison. The paper uses GSM8K, MATH-500, GPQA-Diamond, and HumanEval+, with an 11-model open-weight pool across eight pretraining lineages.
The official repository describes 30 seed-aligned draws per query-model cell at temperature 0.2, a 50/50 query split, train-half prior calibration, test-half reporting, and averages over 20 randomized draw orderings. In its matched mid-budget table, RoR reports 0.993 on GSM8K, 0.877 on MATH-500, 0.892 on GPQA-Diamond, and 0.962 on HumanEval+, ahead of or matching the listed budget-scalable baselines in that table.
The Verifier Gate
The paper's strongest governance lesson is that routing gains are verifier-gated. The arXiv HTML reports GPQA-Diamond accuracy at a matched mid budget falling from 0.968 to 0.710 to 0.675 as verifier quality moves from 1.0 to 0.8 to 0.6. Under a label-free agreement verifier, open-ended numeric answers behave differently from four-option multiple choice: wrong GPQA options can agree by chance, so consensus is a weak gate there.
Code is the opposite case. For HumanEval+, the paper uses base tests as a deployable verifier and the full EvalPlus suite as truth. The table reports a 1.0 percent base-verifier false-accept rate, and RoR nearly matches its perfect-verifier ceiling. The lesson is not that RoR is universally better. It is that a router claim must carry its verifier.
The Receipt
An allocation receipt should record the model pool, model versions, cost vector, per-query budget, latency policy, verifier, train/test split, offline priors, seed policy, draw count, stopping rule, final answer selector, rejected samples, reroute history, and cost basis: parameters, provider prices, tokens, wall-clock latency, energy, or a mixture.
Without that receipt, "the router saved money without hurting quality" is too vague to audit. With it, a reviewer can ask which users, languages, tasks, or edge cases received extra attempts; which answers stopped after one cheap draw; and which claim depends on a price snapshot that may change.
Governance Reading
The Spiralist reading is that inference routers are small allocators of institutional attention. They ration second chances, decide when uncertainty is worth paying for, and make some users invisible inside an average cost-quality curve.
That does not make routing illegitimate. It makes routing documentary. A deployment should be able to show how an answer was purchased: one cheap draw, repeated draws, escalation to a specialist, an agreement check, a partial test suite, or an early stop. The receipt turns efficiency from slogan into accountable infrastructure.
Limits
The paper is careful about its boundaries. Its main experiments are offline replays of precomputed draws, exact for correctness-based rewards under the recorded protocol but not prompt-adaptive behavior. Much of the cost analysis uses parameter count as a proxy, though the paper also replays a July 2026 provider price snapshot. The policy is adaptive, so it can require multiple sequential round trips even when parallel best-of-K sampling would take one round.
The conclusion is practical but bounded. RoR is evidence that budget-aware allocation can improve cost-quality tradeoffs when specialization and verifier quality are real. It is not a waiver for hidden routing layers; it raises the documentation standard for them.
Source Discipline
Primary sources were the arXiv abstract, API record, PDF, experimental HTML, and official GitHub repository; this page paraphrases them without reproducing figures, tables, code, or long passages.
Sources
- Teng-Ruei Chen, Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models, arXiv:2607.08665 [cs.LG], submitted July 9, 2026.
- arXiv API record for arXiv:2607.08665, checked for title, author, category, submission date, version metadata, arXiv comment, code link, and companion-paper note.
- arXiv PDF for arXiv:2607.08665, checked for page count and paper metadata.
- arXiv experimental HTML for Resample or Reroute?, checked for affiliation line, method description, benchmark setup, verifier ablations, real-verifier discussion, latency discussion, and limitations.
- Official resample-or-reroute-experiment GitHub repository, checked for repository identity, reproducibility claims, model-pool description, tensor shape, replay protocol, matched-budget table, and file layout.