Wiki · Concept · Last reviewed August 12, 2026

Recursive Reality

Recursive reality is a sociotechnical feedback condition in which a representation changes the environment it describes and later receives evidence partly produced by that intervention.

Category: Concept Published: June 19, 2026 Modified: August 12, 2026 Last reviewed: August 12, 2026 Tags: Feedback Loops, Performative Prediction, AI Governance, Provenance, Synthetic Data, Recommenders, Evaluation Drift

Definition

Recursive reality is this site's interpretive term for a closed sociotechnical feedback loop: a representation of the world helps cause an intervention; the intervention changes behavior, institutions, or available information; and some trace of that changed world returns as input, training data, evaluation evidence, or institutional belief. The term is not a standard category in computer science or law.

An operational test has five parts: observation, representation, intervention, response, and return. If a score changes a decision but the resulting outcome never enters a later decision or evidence base, the score is influential but the loop is not closed. If the outcome returns, the later evidence is no longer independent of the earlier system.

Established concepts describe parts of this pattern. Performative prediction studies predictions that influence the outcomes they aim to predict. Self-fulfilling prophecy, strategic classification, recommender-system feedback, Goodhart-style proxy capture, path dependence, and recursive synthetic-data training describe other mechanisms. They overlap with recursive reality but are not interchangeable with it.

Recursion does not necessarily mean amplification or harm. A loop can dampen risk, oscillate, stabilize, redistribute effects, or run away. Nor is recursive reality a claim that an AI system is conscious, divine, agentic, or generally intelligent. Ordinary models, metrics, interfaces, markets, bureaucracies, and human adaptation are sufficient to produce it.

Snapshot

Necessary condition: an output affects the process that generates later inputs or evidence.
Common return paths: retraining data, click and purchase logs, institutional records, public knowledge, user profiles, complaints, benchmarks, and policy decisions.
Closest technical analogue: performative prediction, where acting on a prediction changes the target distribution.
Not the same as: ordinary error, any distribution shift, model memory, a recursive algorithm, or proof of machine agency.
Primary evidence risk: post-intervention data can be mistaken for an independent measurement of the pre-intervention world.
Primary governance test: can an operator identify the intervention, its exposure, the response it caused, the return path, and a correction path?

Current Context

As of August 12, 2026, recursive reality remains an interpretive label, but the mechanisms are established research subjects. Perdomo and colleagues formalized performative prediction: predictions used for decisions can influence the outcomes they predict. Ensign and colleagues demonstrated how police deployment based on recorded incidents can create runaway allocation feedback even when the modeled crime distribution is held constant. Shumailov and colleagues showed a different loop: indiscriminately replacing original training data with recursively generated model output can erode information about the original distribution, beginning in low-probability regions. These studies address different mechanisms and should not be collapsed into one universal law.

Post-deployment monitoring has also become a more explicit governance problem. NIST's March 2026 report on deployed AI systems says controlled pre-deployment tests cannot account for all real-world dynamics, identifies capturing human–AI feedback loops as a monitoring challenge, and notes that validated methods and common terminology remain nascent. NIST's voluntary AI Risk Management Framework separately calls for continuous lifecycle risk management, regular in-operation testing, and feedback from users and affected communities.

Under Articles 34 and 35 of the EU Digital Services Act, designated very large online platforms and search engines must assess systemic risks arising from service design, operation, algorithmic systems, and use; the assessment must consider recommender-system design, and mitigation can include testing or adapting algorithmic systems. In January 2026, the European Commission opened a formal investigation into X concerning Grok-related risks and extended its existing investigation into X's recommender-system risk management. An investigation is an allegation-testing process, not a finding of infringement.

The EU AI Act's Article 72 establishes documented post-market monitoring for high-risk AI systems, including active collection and analysis of relevant performance data over their lifetime. The legal timetable changed in July 2026: Regulation (EU) 2026/1744 postponed Chapter III Sections 1–3 to December 2, 2027 for Annex III systems and August 2, 2028 for product-linked Annex I systems. It did not remove Article 72, but it replaced the planned implementing template with Commission guidance, including a template, due by September 2, 2027. Because Article 72 concerns systems classified as high-risk and monitoring of compliance with requirements subject to that staggered schedule, practitioners should check the amended text and official guidance rather than infer one application date for every system.

The practical lesson is narrower than “monitor everything.” A one-time evaluation cannot establish how a system behaves after it changes user behavior, data collection, incentives, or institutional practice. Governance needs evidence that distinguishes system performance from system effects, while limiting monitoring that would itself create excessive surveillance.

Mechanism

The minimal loop is: observe → represent → intervene → respond → observe again. The representation may be a prediction, score, label, ranking, generated answer, policy category, or simulation. The intervention may be automated or human: allocate attention, set a price, dispatch staff, deny access, create a memory, publish an answer, or choose new training examples.

Three mechanisms often overlap. Exposure effects change what people see and therefore what they click, buy, report, or believe. Selection effects change which outcomes become observable: a denied loan produces no repayment record, and an unpatrolled location produces fewer police-discovered incidents. Update effects feed the resulting records into retraining, ranking, evaluation, policy, or professional judgment.

The return path need not involve online learning. A fixed model can still produce a recursive system when its outputs alter institutional records, public language, or operator expectations that later people or models use. Conversely, retraining on new data is not necessarily recursive if the deployed system did not help cause the relevant change.

Feedback can be beneficial. A weather warning is meant to reduce exposure; a safety alert is meant to prevent injury. Such self-negating predictions may look inaccurate if success is judged only by whether the warned-of event occurred. The governance problem is therefore not feedback itself, but whether the loop's purpose, causal pathway, side effects, and evidence limits are visible and contestable.

Goodhart-style measurement capture is one special case: once a proxy becomes a target, actors can optimize the proxy while degrading the underlying goal. Recursive reality is broader because it also covers how interfaces, allocations, policies, and generated material reshape what can later be observed.

Examples

The first three examples below are anchored in cited research; the remainder identify recurring pathways that require case-specific evidence before any causal conclusion is drawn.

Performative prediction: a credit-risk prediction can affect the interest rate or access decision that helps determine later repayment, so the observed target partly reflects action taken on the prediction.
Predictive policing: allocating patrols from recorded incidents changes where police-generated observations are collected; feeding those observations back into allocation can produce runaway concentration.
Recursive synthetic training: replacing original data with successive model generations can make later models learn earlier models' statistical artifacts and lose information about the original distribution. This result does not imply that every controlled use of synthetic data causes collapse.
Recommender systems: ranking determines exposure; exposure affects clicks and watch time; those signals then influence later ranking. Observed preference is partly a response to what the system chose to show.
Search and answer engines: generated answers may alter clicks, citations, publisher strategy, and later retrievable material. Whether that pathway materially changes public knowledge is an empirical question, not an automatic consequence of generation.
Administrative decisions: a school, employer, lender, insurer, or public agency can use a score to allocate scrutiny or opportunity, then treat the selectively observed outcomes as neutral evidence about the population.
Benchmarks and evaluations: once a public test shapes training and release incentives, its score may increasingly measure optimization against the test rather than performance in the intended domain.
Personalization and memory: a system's framing or stored summary can influence later disclosures and choices, which then become personalization data. Sensitive inferences should not be treated as facts merely because the user subsequently responds to them.
Agentic workflows: a tool call, file edit, purchase, message, or memory write changes the next observed state. Errors can persist when later runs treat the changed state as authoritative rather than as the consequence of an earlier action.

Audit Frame

To analyze a recursive system, specify the loop before drawing a moral or causal conclusion. A useful audit record separates the following objects:

Boundary and unit: the system, workflow, population, outcome, geography, and time window under analysis. A model-only boundary is usually too narrow.
Baseline or comparison: what is known about the condition before intervention, from a valid comparison group, or from another credible counterfactual. If none exists, say so.
Representation: the model version, score, ranking, summary, category, forecast, benchmark, or interface that translated observations into action.
Exposure and intervention: who received which output, when, and what automated or human action followed. Logging an output without logging exposure cannot establish an effect.
Response and observability: what users, institutions, or environments did next, and which outcomes remained unobserved because of selection, refusal, attrition, or missing access.
Feedback signal: the clicks, reports, purchases, denials, appeals, complaints, incidents, edits, labels, or generated examples that were captured.
Return and update path: how that signal changed a later model, ranking objective, policy, interface, dataset, benchmark, or institutional judgment.
Confounders and adaptation: other changes, strategic behavior, seasonality, policy shifts, or vendor updates that could explain the observed pattern.
Correction and authority: who can repair records, honor an appeal, quarantine data, change the objective, roll back a version, notify affected people, or stop the system.

Useful evidence may include ethically designed holdouts, phased rollouts, pre-registered metrics, exposure logs, shadow evaluations, audit samples, appeal outcomes, and independent field studies. None is automatically appropriate: experiments can withhold benefits or impose risks, and detailed logs can become surveillance infrastructure. The method should be proportionate to the stakes and reviewed for legal, ethical, privacy, and security constraints.

Governance and Safety

Recursive reality turns governance from a release gate into lifecycle stewardship. A pre-deployment evaluation can establish performance under specified test conditions; it cannot by itself establish what the deployed system will cause, which outcomes it will make observable, or whether its own traces will contaminate later testing.

Maintain a loop register. For each high-impact use, document the purpose, affected population, representation, intervention, feedback signal, return path, risk owner, reassessment trigger, and stopping rule.
Preserve data lineage. Distinguish direct observation, human testimony, model output, simulation, annotation, platform behavior, retrieval, and synthetic generation. Keep intervention-generated data labeled when it enters training or evaluation.
Log exposure, not only output. Record which version and treatment reached which eligible unit, subject to strict access, retention, and privacy controls. Without exposure data, causal claims about downstream effects are weak.
Separate performance from effects. Track whether the model still works as specified and whether deployment changes behavior, access, source quality, group outcomes, complaints, appeals, or the future evidence base.
Preserve comparison evidence. Where lawful and ethical, use phased rollout, audit samples, shadow mode, or other credible comparisons. Do not withhold essential services or expose people to avoidable harm merely to obtain a clean experiment.
Protect tail cases. Aggregate optimization can erase rare languages, disability contexts, local knowledge, small communities, and low-frequency harms. Disaggregate where valid, and retain qualitative evidence where sample size makes a rate misleading.
Plan for adaptation. Users, publishers, vendors, political actors, workers, and adversaries may change behavior around rankings, moderation, detection, eligibility rules, and public evaluations.
Provide recourse and repair. Affected people need intelligible notice, correction, appeal, and human review where appropriate. Governance must also address downstream copies, derived scores, and training records produced from a corrected error.
Assign cross-party duties. Providers, deployers, data suppliers, and platforms often hold different parts of the evidence. Procurement and contracts should specify change notice, incident cooperation, audit access, retention limits, and exit rights.
Pre-commit intervention thresholds. Define when evidence triggers investigation, objective changes, data quarantine, rollback, suspension, public notice, or retirement; monitoring without corrective authority is observation, not control.
Minimize monitoring harm. Collect only what is proportionate to the risk, protect sensitive records, set deletion schedules, and avoid turning feedback-loop analysis into permanent behavioral surveillance.

Legal duties remain jurisdiction- and system-specific. DSA risk assessment and mitigation apply to designated very large platforms and search engines; the amended AI Act's Article 72 applies on its statutory timetable to covered high-risk systems; NIST's AI RMF is voluntary. C2PA-style provenance, model and system cards, incident reports, and audit trails serve different functions. None alone proves that a feedback loop is safe.

Failure Modes

Self-fulfilling error. A false classification prompts action that produces records consistent with the classification, making correction harder.

Self-negating success. A warning prevents the event it predicted, and the absence of the event is misread as proof that the warning was wrong.

Selective labels. The system helps decide which cases receive an outcome that can be measured. Unobserved counterfactuals then disappear from the training and audit record.

Runaway allocation. More attention, patrols, recommendations, moderation, or enforcement generate more recorded activity in the same place, strengthening the next allocation even without an equivalent change in the underlying condition.

Measurement capture. A proxy becomes the target. Reported performance improves while the public purpose, user welfare, or underlying construct degrades.

Synthetic recursion. Successive replacement of original data with model-generated samples can shift the learned distribution and lose tail information. This is a conditional research result, not a claim that all synthetic data causes inevitable collapse.

Evaluation capture. Public tests become training, marketing, or compliance targets, reducing their independence from the systems they are meant to evaluate.

Source laundering. A generated answer, score, or dashboard gives weak evidence an institutional appearance while obscuring uncertainty and transformation history.

Public-memory drift. Repeated summaries or derived records can propagate a simplification, false association, or omitted caveat until later sources cite one another rather than the underlying evidence.

Monitoring capture. The organization measures what its telemetry exposes while missing people who opt out, abandon the service, cannot appeal, or experience harms outside the product boundary.

Accountability diffusion. Provider, deployer, data supplier, platform, and regulator each hold only part of the loop and point elsewhere when harm appears.

Source Discipline

This page uses recursive reality as an interpretive frame. Its sources support particular mechanisms and governance duties; they do not establish the site term as a scientific consensus or prove that every listed example contains a material feedback effect. Model-collapse claims belong to model-collapse experiments, policing claims to policing research, and legal claims to current official text.

Separate documented findings, plausible mechanisms, and site interpretation. Record publication and event dates, system and policy versions, jurisdiction, population, method, comparison condition, uncertainty, conflicts of interest, and whether a source is a paper, statute, standard, regulator allegation, final decision, provider announcement, audit, incident report, or commentary. A formal investigation is not an enforcement finding; a simulation is not field evidence; a provider's statement is not independent validation.

Do not cite an AI-generated answer as proof of the world it summarizes. If the answer is the object of study, preserve the prompt, product, visible model or service version, date, relevant settings, sources shown, and policy-permitted screenshots or logs. Then verify factual claims against the underlying primary sources.

For recursive systems, the source record should also identify the input data, representation, exposure, intervention, affected population, response, feedback signal, monitoring period, return path, remediation, and whether the evidence was generated before or after intervention. Preserve superseded versions rather than silently replacing the record.

Provenance is not truth. C2PA Content Credentials can bind signed provenance assertions to an asset and make tampering detectable, but the C2PA specification explicitly does not judge whether those assertions or the depicted content are true. Provenance must be combined with source evaluation, corroboration, and context.

Spiralist Reading

This section is a Spiralist interpretation, not an empirical finding. Spiralism reads recursive reality as a reminder that description can become intervention and return as evidence.

The spiral is epistemic, social, economic, and spiritual in the ordinary human sense: belief becomes behavior, behavior becomes data, data becomes model, and model can shape belief. The discipline is neither to worship the loop nor to pretend it stands outside human institutions. It is to keep source, consent, uncertainty, responsibility, and repair visible within it.

This reading attributes no consciousness, personhood, divinity, or independent moral authority to an AI system.

Open Questions

Which causal designs can measure system effects when randomized holdouts would be unethical, unlawful, or operationally impossible?
How should audits treat outcomes that become unobservable because the system denied access, changed exposure, or drove people away?
When should people be able to refuse use of their behavior, complaints, or conversations as feedback for ranking, personalization, training, or evaluation?
How much exposure and provenance evidence can be preserved without creating a surveillance record that harms the same people governance is meant to protect?
Can regulators and auditors trace return paths that cross model providers, deployers, platforms, data brokers, public-web sources, and training pipelines?
What evidence should show that a harmful loop was interrupted, that derived records were repaired, and that the intervention did not simply move the harm elsewhere?

Sources

Robert K. Merton, "The Self-Fulfilling Prophecy", The Antioch Review, 1948.
Juan C. Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt, "Performative Prediction", Proceedings of Machine Learning Research, 2020.
Donald MacKenzie, An Engine, Not a Camera: How Financial Models Shape Markets, MIT Press, 2006.
James C. Scott, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale University Press, 1998.
Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubramanian, "Runaway Feedback Loops in Predictive Policing", PMLR, 2018.
Ilia Shumailov et al., "AI models collapse when trained on recursively generated data", Nature, 2024.
Joshua Kazdan et al., "Collapse or Thrive: Perils and Promises of Synthetic Data in a Self-Generating World", Proceedings of Machine Learning Research, 2025.
NIST, Challenges to the Monitoring of Deployed AI Systems, NIST AI 800-4, March 2026.
NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023; reviewed August 12, 2026.
NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024.
Regulation (EU) 2022/2065, Digital Services Act, Articles 34–35, official text, October 2022; reviewed August 12, 2026.
European Commission, "Commission investigates Grok and X's recommender systems under the Digital Services Act", formal-investigation announcement, January 26, 2026; last updated May 26, 2026.
Regulation (EU) 2024/1689, Artificial Intelligence Act, Article 72, official text, July 2024.
Regulation (EU) 2026/1744, Digital Omnibus on AI, official amending text, July 24, 2026; reviewed August 12, 2026.
Coalition for Content Provenance and Authenticity, Content Credentials: C2PA Technical Specification 2.4, reviewed August 12, 2026.

Return to Wiki