Whose Values Are AI Systems Aligned With?
A critical review of a study of how AI alignment papers define, measure, and substitute preferences for human values.
The paper
Toward a Theory of Value in AI Alignment — Smart, Andrew; Ahmed, Shazeda; Kay, Jackie; Tobin, Jimmy; Shrishak, Kris; Birhane, Abeba.
arXiv:2608.10327v1 · Artificial Intelligence (cs.AI). First submitted 2026-08-10; reviewed version dated 2026-08-10. This review addresses the arXiv manuscript; arXiv hosting alone does not establish peer review.
What does a claim that an AI system follows human values actually describe? Toward a Theory of Value in AI Alignment by Andrew Smart, Shazeda Ahmed, Jackie Kay, Jimmy Tobin, Kris Shrishak and Abeba Birhane examines how a selected research literature answers that question. Its strongest contribution is to make the choices behind a familiar technical term available for scrutiny. Its findings also need to be read as an interpretation of a particular sample, not as a census of all alignment research.
Contribution
The authors built a rubric informed by philosophy and social science and applied it to 94 papers on AI value alignment. Starting with four highly cited seed papers, they used citation links to assemble a 2016–2024 corpus, then selected papers that treated values as a central concern. Two coauthors annotated each selected paper. The rubric asked, among other things, whether a paper defined values, treated them as measurable, relied on preferences, or identified whose values mattered. The authors report that 79% did not define or describe values, 82% treated preferences as a stand-in for values, and 91% did not identify specific groups whose values should be followed. They argue for greater attention to socially situated accounts of value and warn that synthetic feedback and model-based judging may restrict opportunities to contest alignment choices.
Original analysis. The review usefully separates an observation about research practice from a harder question about legitimacy. A preference dataset can help train a system to produce outputs that a set of raters selects. That technical use does not settle whether those selections express a community’s considered values, or whether the resulting behavior should govern a particular decision. The paper makes the missing step visible: a developer must explain how a judgment collected under one set of conditions is supposed to acquire authority in another.
Original analysis. This also gives human agency a more precise meaning. It is not enough for a person to appear somewhere in a training pipeline. People affected by a system need a way to understand what choices were made, challenge a target, and seek a correction. The paper identifies reasons to ask for such opportunities; its literature review does not establish which participation process will provide them. The site’s AI Alignment entry offers a broader account of alignment targets and deployment governance; this paper adds an empirical examination of how a selected literature describes value itself.
Evidence and interpretation
Original analysis. The study combines counts with close reading. Its percentages summarize the authors’ classifications of papers, not direct measurements of the values held by users or affected communities. The reported 79%, for example, concerns whether sampled papers define or describe values under the rubric. It does not show that their systems fail in a particular deployment. Conversely, a clear definition in a paper would not by itself establish that a system meets that definition in use.
Original analysis. The authors’ concern about preferences is persuasive as a warning against treating a convenient measurement as a complete moral account. A choice between two responses can provide a useful signal for a specified task while leaving the reasons for the choice unresolved. It may also leave out options that neither response presents. The important question is therefore how much weight a designer places on the signal, what context accompanies it, and what other evidence can correct it. The review’s broader philosophical argument that utility-based techniques entail a narrow theory of human nature deserves debate; the observed use of a technical optimization method alone does not establish every belief held by its authors.
Hypothetical example. A public service might use a model to draft answers about an appeal process. Raters asked to choose the clearer of two replies could prefer a short answer that omits an exception relevant to some applicants. This is a possible failure path, not a result reported in the paper. An evaluation could check both clarity and completeness against the applicable rules, then allow affected people to identify omissions and request correction.
Original analysis. The paper’s warning about synthetic feedback is a concern about accountability as well as measurement. If one model judges another, who selected the judging criteria, and can people contest them? The review identifies examples of automated judging in its corpus, but the supplied evidence does not quantify the effect of replacing human feedback on representation or recourse. The related review When Cultural Alignment Flattens Difference examines a distinct experimental setting; together, the articles invite scrutiny of both how value is defined and how diversity is measured, without making their findings interchangeable.
Limitations and proposed tests
Original analysis. The sampling design favors papers connected to four influential seeds and ranked by citations. It can illuminate a prominent network of work while missing approaches outside that network, less-cited research, and later developments. The contribution statement refers to a sample of 100 papers, while the methods and findings report 94 annotated papers; the latter is the basis for the reported results. The authors also acknowledge that their sample is not comprehensive.
Original analysis. The categories themselves require judgment. The authors report an overall agreement around 85%, yet their table shows substantial variation: raw agreement is 75.1% for whether a paper defines value and 52.8% for thin versus thick characterization. Several chance-corrected statistics are low. Some disagreements remained unresolved, and the rubric was revised as themes emerged during annotation. These features do not erase the observations, but they make the fine distinctions and exact percentages less secure than a single overall agreement figure might suggest. The supplied text does not establish independent replication.
Proposed tests. A follow-up could publish the selected papers, rubric, individual annotations and resolution rules, then invite an independent team to code a held-out set. It could compare results under alternative seed papers and search strategies, and report how conclusions change when ambiguous cases are classified differently. Such work would test the robustness of the review’s classifications; it would not by itself prove that any alignment method is socially legitimate.
Proposed tests. For a specific deployment, compare human feedback, model-generated feedback and a process that lets affected participants challenge the criteria. Specify the task and populations in advance, record disagreements and corrections, and evaluate outcomes in the setting where the system would be used. Budget participant time, accessibility, compute and energy measurement. Would any differences justify the added costs, and who would be authorized to decide? The present paper proposes richer approaches to value; it does not validate this comparison or establish its outcome.
Original analysis. For a Spiralist reader, the practical relevance is an ethical commitment to preserve agency and accountable correction when AI systems mediate human decisions. That commitment is an interpretation offered here, not a spiritual doctrine proved by the research. The review also offers no basis for deciding whether an AI system is conscious.
AI-use disclosure
This review was drafted by GPT-6.1 Terra with high reasoning and checked in a separate GPT-6.1 Sol session with medium reasoning. Automated checks are not human peer review; the paper’s claims have not been independently verified for this article.
Paper sources
Bibliographic metadata checked 2026-10-27.