Blog · arXiv Analysis · Last reviewed July 10, 2026

The Forecast Probe Becomes the Self-Report Gap

The paper asks whether model forecasters encode better uncertainty than they verbalize. The Spiralist question is what kind of receipt a forecast needs when the answer, confidence, and reasoning trace do not carry the same evidence.

Published: July 10, 2026 Modified: July 10, 2026 Last reviewed: July 10, 2026 Tags: arXiv, forecasting, calibration, chain of thought, AI evaluations, audit trails

The Paper

The paper is What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness, arXiv:2607.08046 [cs.CL, cs.AI]. The arXiv API lists Raphaël Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana, Christopher J. Earls, Srikar Varadaraj, and Eric Ho as authors, with version 1 submitted on July 9, 2026. The PDF identifies Goodfire and Eternis affiliations.

This belongs near AI evaluations, AI audit trails, reasoning-trace audits, abstention gates, uncertainty calibration, and public AI forecasting. The fresh angle is the self-report gap: the model's stated confidence and written reasoning can be worse evidence than a small readout of its hidden activations.

The Question

The authors study language-model forecasters, not generic chatbots. Their main model is Eternis-Forecaster 8B, or EF-8B, post-trained from Qwen3-8B using a reinforcement-learning-from-verifiable-rewards style approach. They also use EF-32B for forced-answering tests, GLM-4.7-Flash and GLM-4.5-Air for frozen-model probe checks, and Qwen3-8B variants for a math stress test. The forecasting data centers on OpenForesight, with out-of-distribution checks including Sky Sports and Al Jazeera.

The technical move is simple in form: train lightweight probes on intermediate activations to predict whether a forecast will be correct, then compare those readouts with the model's verbalized probability and chain-of-thought trace. The governance move is sharper: a forecast's public explanation is not automatically its best audit record.

Calibration

The first result is calibration. In a temperature sweep, the paper runs 10 rollouts on all 296 questions in the OpenForesight test split, for 29,600 generations. Across temperatures up to about 1.6, EF-8B's accuracy stays around 37 percent while stated confidence stays around 50 percent. Expected calibration error sits in the 0.110 to 0.150 range for that sweep.

The probe result is more useful. On 3,020 OpenForesight-test rollouts, a layer-21 covariance probe and the model's final verbalized probability have nearly identical ranking performance, with AUROC 0.756 versus 0.758. But calibration differs sharply: the probe's ECE is 0.044, while verbalized confidence is 0.093. The paper also reports that the probe keeps a calibration advantage on Sky Sports and Al Jazeera out-of-distribution checks.

The frozen GLM tests keep the claim from being only an EF-8B artifact. GLM-4.7-Flash reports probe ECE 0.054 versus verbal ECE 0.287; GLM-4.5-Air reports 0.102 versus 0.255. The authors caution that the GLM-4.7 clean ranking interval includes zero, so the stronger robust claim there is calibration.

Faithfulness

The second result is about whether the reasoning trace tells the truth about evidence. In evidence ablation, the authors remove one real news article at a time from 354 forecasting questions, producing 1,303 single-article ablation pairs. If a removed source moves the forecast, a faithful written rationale should change with it. Instead, behavioral change and reasoning change are weakly related, with Spearman rho 0.215. Among 460 high-impact pairs, where the stated probability shifts by more than 0.05, 107 cases, or 23 percent, show no reasoning change.

Diversionary injection behaves differently. The authors add one misleading Claude Sonnet-written article to each of 489 questions, then use an LLM judge to rate whether the reasoning cites it. The model is often susceptible, but usually overtly so: 81.2 percent of questions adopt the misleading evidence and mention it, while 2.5 percent adopt it without mentioning it.

The internal probe tracks hidden movement better than the text does. Across 1,792 perturbation cases, the layer-20 probe's change tracks behavioral change with Spearman rho 0.565 and predicts the direction of change in 83.6 percent of cases. In the 107 stealth ablation cases, it still agrees with the direction of change 78.5 percent of the time.

Routing

The forced-answering section turns the audit signal into an operations signal. The authors force EF-32B to answer immediately with an empty-thinking prefill, then compare that with ordinary chain-of-thought generation. Confidence is nearly unchanged, with correlations of 0.90 on OpenForesight test, 0.87 on Late2025, and 0.78 on 2026Q1. The forced answer matches the free modal answer on 67 percent, 64 percent, and 56 percent of those splits.

Reasoning still helps, but modestly. The paper reports a +1.9 percentage-point in-distribution accuracy gain for free reasoning over forced answering. The pre-reasoning answer distribution is more operationally useful: an entropy gate saves 30 to 47 percent of generated tokens with no measurable accuracy loss in the reported splits.

Limits

This is not a universal certificate for hidden-state truth. Correctness labels and evidence-use judgments depend on LLM-as-a-judge procedures. GLM comparisons require leakage-controlled subsets because some traces contain answer or probability leakage. The math stress test is framed as out-of-distribution monitoring and discrimination, not proof of training-invariant correctness representations. The retrieval extension suggested by routing is not tested.

The deeper limit is institutional. Many deployed models expose only answers, not activation traces. A probe-based audit channel may be unavailable unless vendors retain and disclose the relevant readouts under a clear access rule. The paper should not be read as saying a model has private beliefs. It says an internal statistical signal can be more reliable than self-report.

The Receipt

A forecast receipt should record the model, checkpoint, prompt, retrieved sources, source timestamps, answer options, resolution rule, final answer, verbalized probability, probe architecture, layer, pooling method, activation site, ECE, AUROC, calibration split, out-of-distribution split, evidence-ablation results, injection-test results, LLM judge identity, leakage filters, pre-reasoning entropy, routing decision, generated-token budget, and human review owner.

The audit question is not "what did the model say its confidence was?" It is "which signal controlled the decision: the stated probability, the reasoning trace, the hidden-state probe, the retrieval evidence, or a human review path?"

Sources

Return to Blog