Measuring Perceived Alignment Without Mistaking It for Alignment
A critical review of the Perceived Value Alignment Scale, its evidence base, and the limits of treating user perception as proof of alignment.
The paper
A Scale For Value Alignment In Human-AI Interaction — Hegemann, Lena; Villa, Steeven; Bang, Hyemin; Gordon, Mitchell L.; Oulasvirta, Antti; Welsch, Robin; Ebel, Patrick.
arXiv:2610.09911v1 · Human-Computer Interaction (cs.HC). First submitted 2026-10-07; reviewed version dated 2026-10-07. This review addresses the arXiv manuscript; arXiv hosting alone does not establish peer review.
Contribution
Paper synopsis
Hegemann et al. define perceived value alignment as a person’s judgment, after interacting with and observing an agent, of congruence among values, goals, and preferences. They build the 13-item Perceived Value Alignment Scale (PVAS) from prior human-AI questionnaire items, expert screening, and vignette studies. Seven experts narrowed 274 candidates to 106; an English-speaking US/UK item study produced 518 post-screening respondent-scenario rows for exploratory factor analysis. The extraction used varimax rotation, which the authors acknowledge imposes uncorrelated factors. A separate sample supported confirmatory and validity analyses. The final scale has Value Understanding (eight items) and Value Manifestation (five); confirmatory analysis removed a direct influence item when it loaded weakly. Reported model fit was good, internal consistency was high (α=.905 for manifestation and .890 for understanding), and planned driving and travel vignettes separated scores, especially manifestation. The six-item short form was optimized on pooled development and validation data. Evaluation relied on vignettes rather than functional systems, sampled English speakers in the US and UK, and did not independently validate the short form in a standalone study. These results support a measure of perception in these settings, not proof of objective alignment. See the paper’s construct definition, item development, expert review, scenario study, factor analysis, validation, and limitations.
Original editorial analysis
The paper’s useful move is to turn a broad alignment question into a measurement question about users’ experience. Separating perceived understanding from perceived manifestation may help researchers see whether a system seems to grasp what matters to someone and whether its actions appear to reflect that. This gives interaction studies a reusable outcome to compare across designs. It does not make the outcome a certificate: the score remains tied to a person’s judgment after a particular interaction and cannot, by itself, establish what an agent will do in a different situation.
The scale is not a direct measure of human agency. The removal of the item about influencing the system is a telling boundary: the final score does not directly ask whether a person can veto, revise, pause, or reverse an action. A person could report feeling understood while lacking control over what an agent does; whether that combination occurs in practice is a question for further study. Researchers evaluating delegated systems should therefore report perceived alignment alongside evidence about permissions, review points, correction paths, and who remains accountable when the agent acts.
Evidence and governance implications
Original editorial analysis
A high PVAS score would be evidence that participants perceived alignment in the tested interaction. It would not demonstrate that the system followed their values across untested cases, that those values are ethically legitimate, or that other affected people would agree. This distinction matters for governance: a system may satisfy its user while imposing costs on a colleague, customer, patient, or member of the public. A common measure can help compare user experience, but it cannot settle whose preferences should govern a shared decision. Researchers could report individual scores and disagreement across affected groups instead of relying only on a pooled average; whether that approach improves evaluation is itself testable.
The word understanding also needs careful handling. The Value Understanding items ask users to judge whether an agent understands them, but that judgment is not evidence of machine consciousness or subjective experience. The paper measures perception through self-report; it does not test consciousness or support spiritual doctrines. For a governance review, perceived understanding should be kept distinct from evidence about the agent’s decisions, the reasons it presents, and how a person can challenge or correct those decisions.
Hypothetical example
Consider a hypothetical scheduling agent that appears to understand a manager’s preference for consistency and applies it to staff requests. The manager might rate its understanding highly, while an employee affected by a decision might see the outcome as unfair or might not have a route to appeal. This example is not a result from the paper. It illustrates why a user-perception score should be paired with questions about whose values were elicited, who is affected, and which institution or person can answer for the result.
Proposed tests
The following are proposals, not findings reported in the study:
- Compare vignette ratings with ratings after multi-turn use of a functional system. A follow-up could test whether the two factors retain their structure when users can correct the system and observe later behavior.
- Vary user control in otherwise matched tasks, such as offering a preview, veto, edit, or undo option. The study could test whether those changes affect perceived understanding, perceived manifestation, or both.
- Present the same decision to multiple affected parties and examine score distributions alongside outcome and appeal records. This could show where a single user rating masks disagreement, without assuming that an average represents a shared value.
- Test for impression effects by comparing a system optimized to raise PVAS ratings with its performance on new, independently assessed scenarios. The question is whether ratings continue to track actions users endorse when the system has an incentive to appear understanding.
- Evaluate the short form in a standalone study and across additional languages and cultural settings, checking whether item meanings and factor structure remain comparable.
PVAS also leaves operational and material costs outside its scope. It does not measure the computing, energy, labor, or hardware resources required to produce a given user experience. A deployment assessment that considers those costs would need separate evidence rather than treating a favorable perception score as a complete account of system value.
Limitations
Original editorial analysis
Vignettes allow researchers to present controlled contrasts, but they cannot reproduce the uncertainty, accumulated expectations, and changing priorities that may arise in sustained delegation. A useful next step would test whether the scale remains meaningful when users can observe an agent repeatedly, correct it, and decide whether to delegate again. The reported results should also be read within the study’s language and sampling boundaries; broader use requires evidence that the instrument works for people and contexts not represented there.
The two-factor structure is a useful working model, not proof that these are the only dimensions of alignment or that they are always separate. The paper defines values, goals, and preferences as distinct levels, while its measured factors divide perceived understanding from action. That difference invites further theory and testing: similar actions might reflect different priorities, and different actions might reflect the same priority under different constraints. Follow-up work could examine whether the items distinguish perceived value congruence from competent execution, persuasive explanation, or social fluency.
For a broader account of what an AI evaluation can and cannot establish, see AI Evaluations. For the governance question of who defines an alignment target, see AI Alignment. These readings complement the paper’s narrower contribution: measuring perceived alignment in human-AI interaction.
AI-use disclosure
This review was generated and checked by separate GPT-6 Luna runs with maximum reasoning. Automated checks are not human peer review; the paper’s experiments have not been independently replicated for this article.
Paper sources
Bibliographic metadata checked 2026-10-23.