Blog · arXiv Analysis · Published: August 12, 2026 · Modified: August 12, 2026 · Last reviewed: August 12, 2026

The Self-Feeding Probe Becomes the Apparatus Test

Nicolás Vera Zúñiga asks what an experiment measures when a language model repeatedly consumes its own output.

An apparatus-test receipt separates model behavior from behavior created by the loop, estimator, sampling rule, and experimental geometry.

The Paper

The paper is Nicolás Vera Zúñiga's What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the model, arXiv:2608.10986v1 [cs.CL]. arXiv records submission on August 11, 2026 and describes a 16-page paper with four figures. The PDF lists Vera Zúñiga as an independent researcher in Chile.

The paper groups self-consistency, iterative refinement, and agentic loops as methods that derive readings from model-produced material. Refinement and agentic workflows can return that material as new input, while self-consistency samples and aggregates it. The observed result belongs to a larger construction: model, prompts, memory, update rules, sampling, stopping, aggregation, and evaluation.

This differs from the site's grading-cascade essay. There, an evaluator can misread an agent's answer. Here, the apparatus can help produce the phenomenon being measured.

The Closed Loop

Vera Zúñiga constructs a ring of token cells. Each cell is repeatedly resampled from the model's conditional distribution over a local window, at a chosen radius and temperature. Two rings begin identically except for one flipped token and then receive the same random numbers and visit order. Where their windows remain equal, they draw the same token exactly; later disagreement can therefore be traced to the original perturbation.

The author explicitly treats this as a probe, not a deployment model. Ordinary autoregressive generation appends tokens instead of revisiting them. In the paper's check on three Pythia models, an injected token difference persisted through every tested continuation. The ring's ability to absorb or spread a difference is first a property of in-place resampling.

A Real Result About the Wrong Object

At low temperature, the autoregressive ring shows a sharp transition into a nearly frozen state. For Pythia-410m, 81 of 96 cells become the newline token at the lowest reported temperature. The transition is reproducible and statistically structured, but the paper's controls show that it belongs to the probe.

The mechanism is an attracting fixed point in the low-temperature update map. A beginning-of-sequence prefix sharply reduces the collapse without changing model weights; increasing the radius moves the system out of the narrow regime; and masked-language-model controls do not show the transition. Tests across 19 models and scale ladders spanning 70-fold do not turn it into a general model comparison. Precision did not identify the object. Intervention did.

The Discriminator

The paper's central test is a two-direction manipulation. Hold the model fixed and vary the construction. Then hold the construction fixed and vary the model. A reading that follows the apparatus but not the model is construction-determined. A reading that moves with the model while the apparatus stays fixed has earned a narrower model claim.

The damage light cone and the radius scaling of the token-space growth exponent fail as cross-model signals: geometry largely fixes them. Yet the same instrument is not empty. With the construction unchanged across Pythia training checkpoints, the exponent crosses zero reproducibly. The paper therefore treats it as a within-model developmental quantity, not a model-ranking metric.

Another readout, the dominant token's share of the settled ring, preserves model ordering across six tested constructions much better than the exponent does. That contrast is the point. One apparatus can produce valid evidence for one comparison and nearly no information for another, even when both outputs have clean units and stable decimals.

The Estimator Must Earn Its Number

The paper reports four retracted verdicts caught by known-answer systems rather than ordinary review. Among the failures were calibration at a different geometry, correlated replicas caused by sharing one visit order across a batch, a test never shown capable of separating its hypotheses, and treating an undefined exponent as zero. One shared-order error made uncertainty estimates roughly eight times too small and inverted a verdict.

The released findings ledger keeps amendments and retractions in place. That is stronger than quietly replacing a graph. It shows how a plausible measurement changed status, which guard caught it, and what remained valid. The repository and Zenodo archive also preserve code, per-run results, figure scripts, and an estimator-gate package.

The Governance Reading

For AI governance, the loop is part of the evaluated system. A high agreement rate may belong to the aggregation rule. Fast convergence may belong to the stopping criterion. Apparent self-correction may depend on critic prompts, retries, memory, or how failures are discarded. None of those possibilities proves a metric invalid. Each makes attribution an experimental obligation.

The paper has not performed this separation for deployed self-consistency, refinement, or agent workflows; it identifies that extension as open work. The responsible lesson is a testable question: does the claimed behavior survive when the loop changes while the model stays fixed, and does it move when the model changes while the loop stays fixed?

The Apparatus-Test Receipt

An apparatus-test receipt should record the model and checkpoint, tokenizer, loop topology, initial state, context or window, update schedule, temperature, sampling rule, random coupling, visit order, perturbation, seed, independent unit, stopping rule, estimator, undefined-case policy, comparison geometry, control system, and software hash.

It should also preserve the original claim, calibration result, construction variations, model variations, failed controls, retractions, amendments, and final scope. A number without that record can be reproducible while its interpretation remains wrong.

Limits

This is one preprint built around a deliberately artificial token lattice. The developmental crossing is measured within Pythia; other public model families do not provide checkpoints inside the relevant training window. The component-ablation work uses one family and one radius, and the paper does not claim interpretability or a general recipe for predicting failures.

The finding is methodological, not mystical: self-feeding systems join model and apparatus. Separating their effects requires interventions on both. It does not show that every loop artifact is useless, that every agent evaluation is confounded, or that the measured model-dependent results will transfer beyond the reported conditions.

Source Discipline

Primary sources were the arXiv abstract, metadata API, HTML, complete version 1 PDF, public code repository, findings ledger, and Zenodo archive. This essay reports the author's experiments and corrections without reproducing figures, tables, proofs, code, or extended passages; it does not independently rerun the released package.

Sources

Return to Blog