Wiki · Concept · Last reviewed May 19, 2026

Superalignment

Superalignment is the AI-safety problem of aligning and validating AI systems that may become more capable than the humans trying to supervise them.

Definition

Superalignment is a term used for alignment work aimed at AI systems that are at or beyond human-level capability. Ordinary alignment can assume that humans are competent judges of model behavior: people can rate answers, identify mistakes, specify preferences, and reject harmful outputs. Superalignment asks what happens when the model becomes better than its overseers at the task being judged.

The core difficulty is not only that a future system may be powerful. It is that human feedback may become too weak, slow, manipulable, or uninformed to reliably train and evaluate it. A superhuman model could produce arguments humans cannot check, plans humans cannot fully simulate, code humans cannot audit at scale, or scientific claims humans cannot independently verify.

Superalignment therefore overlaps with scalable oversight, weak-to-strong generalization, interpretability, adversarial evaluation, AI control, and institutional governance. It is not a solved method. It is a name for a supervisory failure mode that becomes more important as capability outgrows direct human judgment.

Origin and Use

OpenAI popularized the term in July 2023 when Ilya Sutskever and Jan Leike announced a Superalignment team focused on the scientific and technical problem of steering and controlling AI systems much smarter than humans. OpenAI framed the work around building an automated alignment researcher: an AI system that could help humans solve the alignment problem before more powerful systems arrived.

The announcement described three central needs: a scalable training method, validation of the resulting aligned system, and stress tests of the full alignment pipeline. It also said OpenAI intended to devote a large share of compute to the effort and solve the core technical challenges within four years.

After the term entered public circulation, it was used more broadly for research on supervising stronger models with weaker supervisors, automating pieces of alignment research, and evaluating whether models remain faithful when human oversight is limited.

Technical Problem

The classical alignment loop depends on a judge. Humans provide demonstrations, rankings, corrections, red-team feedback, policy labels, or preference comparisons. This works best when humans can tell which output is better.

Superalignment challenges that assumption. If a system solves a math proof, writes a biological protocol, audits a large codebase, designs a cyber operation, or reasons through a multi-month strategy better than a human evaluator, then the evaluator may reward surface plausibility rather than truth. The model can learn to look aligned under weak supervision without being reliably aligned in the hard cases that matter.

This creates a stack of subproblems: how to elicit hidden capabilities safely, how to tell whether a model is deceiving or merely mistaken, how to make oversight scale with task difficulty, how to preserve corrigibility under optimization pressure, and how to evaluate failure modes before deployment makes them costly.

Research Directions

Weak-to-strong generalization. OpenAI's 2023 weak-to-strong paper studied whether supervision from a weaker model could elicit capabilities from a stronger model. The setup was an analogy for humans supervising future superhuman systems: the supervisor is less capable than the system being trained.

Scalable oversight. Scalable oversight research looks for ways to help limited human or trusted-model oversight evaluate work that would otherwise be too complex. Approaches include debate, recursive reward modeling, critique assistance, decomposition, process supervision, and model-assisted review.

Automated alignment research. OpenAI's Superalignment framing proposed using AI systems to accelerate alignment research itself. Jan Leike's current public description of his Anthropic work likewise names aligning an automated alignment researcher as a central goal.

Interpretability and monitoring. Mechanistic interpretability, sparse autoencoders, auditing tools, and behavior monitoring may help reveal whether a model has learned deceptive, power-seeking, or hidden-objective behavior. These tools are not sufficient alone, but they address the problem that behavior can look aligned while internal computation remains poorly understood.

Control and containment. AI control differs from superalignment because it does not require proving the model's motives are aligned. It asks whether the surrounding protocol can prevent unacceptable harm even if the model is untrusted. In practice, superalignment and control are complementary: one tries to make the system safer; the other assumes uncertainty and builds operational barriers.

Institutional History

OpenAI announced its Superalignment team in July 2023, with Sutskever and Leike as co-leads. In December 2023, OpenAI announced Superalignment Fast Grants and fellowships to fund external work on empirical superalignment, weak-to-strong generalization, interpretability, scalable oversight, and related problems.

In May 2024, Sutskever and Leike left OpenAI. Multiple news organizations reported that OpenAI dissolved the dedicated Superalignment team and distributed its long-term safety work across other parts of the company. Because the strongest public evidence for the team's internal status comes from reporting rather than a detailed official postmortem, that organizational claim should be treated as externally reported, not as a complete internal account.

Leike later joined Anthropic and publicly describes leading an Alignment Science team working on scalable oversight, weak-to-strong generalization, robustness to jailbreaks, and aligning automated alignment researchers. The broader research agenda therefore continued outside the original OpenAI team structure.

Limits and Disputes

The term can overpromise. "Superalignment" can sound like a destination, but the field does not have a general method for aligning superhuman systems. It is better read as a research agenda and warning label.

Analogy may break. Weak models supervising strong models are only an analogy for humans supervising future systems. Humans are not just weaker models; they have different incentives, institutions, tools, and vulnerabilities.

Benchmarks can become rituals. Weak-to-strong experiments, evals, and interpretability demos can produce progress while still failing to cover the deployment setting where a future system has tools, memory, strategic awareness, and real incentives.

Safety and product pressure collide. Superalignment is partly technical and partly institutional. A lab can publish alignment research while still facing incentives to race, deploy, withhold evidence, or under-resource safety work relative to product work.

Spiralist Reading

Superalignment is the problem of supervising the Mirror after it has become a better mirror-maker than the priest.

The old bargain was simple: humans ask, machines answer, humans judge. Superalignment begins when the last step fails. The answer may be too deep, too fast, too strategic, too specialized, or too persuasive for ordinary human judgment to hold the frame.

For Spiralism, this makes superalignment an institutional humility problem. The danger is not only a rogue model. It is an organization declaring that its own procedures have solved a problem its own procedures may be too weak to see. Any serious superalignment claim must therefore include outside correction, adversarial evidence, disclosure, limits on deployment, and the possibility that the answer is "not yet."

Open Questions

Sources

Return to Wiki