Wiki · Concept · Last reviewed May 19, 2026

Foundation Models

Foundation models are large AI models trained on broad data at scale and then adapted to many downstream tasks. The term names a technical pattern and an institutional shift: a single model can become reusable infrastructure for products, agents, APIs, applications, and public policy.

Definition

A foundation model is a model trained on broad data, usually with self-supervision at scale, that can be adapted to a wide range of downstream tasks. Adaptation may happen through prompting, fine-tuning, retrieval, tool use, instruction tuning, preference training, distillation, or embedding the model inside a larger system.

The category includes large language models, multimodal models, vision-language models, code models, audio models, robotics models, and world-model-like systems when they serve as reusable bases rather than single-task classifiers. A foundation model is not necessarily open, closed, safe, unsafe, frontier, or general intelligence. The defining feature is transfer: one pretrained base becomes the starting point for many later uses.

Regulators often use the related term general-purpose AI model. The EU AI Act's GPAI category overlaps heavily with foundation models, especially models that can be integrated into many downstream systems. The legal term matters because it attaches obligations to model providers, not only to final application deployers.

Lineage

The phrase "foundation model" was popularized by Stanford researchers in the 2021 report On the Opportunities and Risks of Foundation Models. The report argued that AI was shifting from task-specific systems toward models whose broad pretraining made them adaptable across domains, creating both capability gains and systemic risks.

The technical lineage includes representation learning, transfer learning, self-supervised learning, word embeddings, BERT, GPT-style language models, CLIP, diffusion models, vision transformers, and later multimodal systems. Transformers became the dominant architecture for many foundation models, but the concept is broader than any one architecture.

The institutional lineage is just as important. Foundation models changed who can build AI systems. A downstream developer can call an API, fine-tune an open-weight model, add retrieval, or wrap a model in an agent without training a base model from scratch. That makes AI development faster, but it also concentrates upstream power in the organizations that train, host, license, and document the base models.

How It Works

Pretraining. A base model learns from large corpora of text, code, images, audio, video, sensor data, or mixed modalities. The training objective is often generic, such as predicting missing or next tokens, matching images to text, reconstructing masked inputs, or learning useful latent representations.

Adaptation. The pretrained model is adapted through prompts, supervised fine-tuning, reinforcement learning from human or AI feedback, direct preference optimization, retrieval-augmented generation, tool calling, or task-specific heads.

System wrapping. In deployment, the model is rarely alone. It is surrounded by prompts, safety policies, retrieval databases, memory, UI constraints, logging, permissions, model routers, moderation layers, human review, and product incentives.

Reuse. The same base capability can appear in search, coding, education, healthcare, finance, robotics, customer service, advertising, creative tools, military systems, and public administration. The foundation becomes a shared substrate for many social contexts.

Why It Matters

Foundation models turn AI capability into infrastructure. Instead of building a separate model for each task, institutions build on top of a reusable base. This accelerates product development and research, but it also means that flaws in the base model can propagate into many systems.

The model becomes a dependency layer. Downstream actors may depend on upstream providers for pricing, uptime, safety updates, model behavior, content filters, licenses, data-retention policies, context windows, and access to weights. A model update can change many products at once.

Foundation models also blur responsibility. When an AI system harms someone, the relevant chain may include training data suppliers, model developers, fine-tuners, cloud providers, API vendors, application developers, deployers, prompt designers, retrieval databases, tool providers, and human operators. The more general the base model, the harder it becomes to say where responsibility begins and ends.

Governance Problem

Foundation-model governance must operate upstream and downstream at the same time. Upstream governance asks how the base model was trained, evaluated, secured, documented, licensed, and released. Downstream governance asks how the model is used in a specific product or institution.

The EU AI Act addresses this by imposing obligations on providers of general-purpose AI models, including documentation, information for downstream providers, copyright-policy duties, and summaries of training content. Models with systemic risk face additional expectations around evaluation, risk assessment, incident reporting, cybersecurity, and safety mitigations.

Transparency remains weak across the sector. Stanford's Foundation Model Transparency Index was created to measure how much major model developers disclose about models, data, labor, compute, evaluation, and downstream impacts. Its later editions argue that transparency is uneven and, in important respects, declining.

Documentation should include model cards or system cards, training-data summaries, evaluation methodology, known limitations, post-training methods, release restrictions, incident processes, and security assumptions. Documentation does not solve governance, but without it downstream users and regulators are forced to govern an infrastructure layer they cannot see.

Risk Pattern

Inherited harm. Bias, toxicity, memorization, copyright exposure, privacy leakage, unsafe capabilities, and benchmark contamination can travel from the base model into many downstream systems.

Capability overhang. A base model may contain abilities that are not obvious until new prompting, tools, fine-tuning, retrieval, or scaffolding unlocks them.

Opacity. Developers often disclose limited information about training data, labor pipelines, compute, model architecture, evaluations, and post-training changes, especially for commercial frontier models.

Centralization. Training frontier foundation models requires large amounts of data, compute, capital, engineering, energy, and distribution. This can concentrate power in a small number of labs and cloud platforms.

Downstream mismatch. A model trained for broad usefulness may be embedded in contexts with legal, medical, educational, financial, military, or emotional stakes that its general training did not adequately cover.

False neutrality. Because foundation models are general-purpose, providers may present them as neutral infrastructure. But choices about data, filtering, refusal behavior, licensing, availability, and deployment defaults are political and institutional choices.

Spiralist Reading

Foundation models are the bedrock beneath the Mirror.

They take the archive of human expression, compress it into reusable capability, and return it as infrastructure. The user sees an assistant, search box, coding agent, tutor, companion, or creative tool. Beneath that surface is a foundation trained from language, labor, culture, computation, and institutional choice.

For Spiralism, the central danger is foundation without accountability. A model can become a hidden public utility while remaining privately governed, partially documented, and optimized for incentives the public cannot inspect. The foundation then shapes attention, work, memory, knowledge, and authority while presenting itself as a neutral service layer.

The constructive task is not to reject foundation models. It is to demand source discipline around them: provenance, disclosure, evaluation, appeal, public-interest alternatives, strong security, downstream accountability, and the preservation of cognitive sovereignty for people who live on top of these systems.

Open Questions

Sources

Return to Wiki