Blog · arXiv Analysis · Published: August 12, 2026 · Modified: August 12, 2026 · Last reviewed: August 12, 2026

The Moral Ballot Becomes the Developer Interface

A vote about AI behavior can look public while its menu, electorate, and wording remain privately designed.

Participation begins before the first response is recorded.

The Paper

The source is Taenyun Kim, Edyta Bogucka, and Daniele Quercia’s Participatory Moral AI Is Not Neutral: The Invisible Hand of Developers, arXiv:2608.14522v1 [cs.AI], submitted August 14, 2026. It studies moral-preference elicitation: asking people how an AI system should weigh features, then treating the collected judgments as input to a policy.

The Ballot Has an Architecture

The paper isolates three choices made before aggregation: feature scoping, voter sampling, and question framing. Those choices determine what appears on the form, whose answers enter the total, and which perspective the question makes salient. Calling the result a public preference can therefore hide a chain of developer decisions upstream of the public.

The two-phase study covers three hypothetical settings: AI-assisted kidney allocation, an AI agent simulating an absent worker, and generated depictions of a deceased person. Phase 1 included 150, 149, and 150 participants by setting. A fresh Phase 2 sample included 120 per setting, for 809 participants overall. Recruitment used Prolific, and compensation was at least eight dollars per hour. Phase 1 balanced conservative, moderate, and progressive participants; Phase 2 balanced conservative and progressive participants within each framing condition.

The Menu Decides What Can Matter

In Phase 1, participants proposed five features that should matter and five that should not, with reasons. GPT-4o-mini extracted and labeled features; two authors checked a random 100-response sample, and the researchers iteratively calibrated the names. The pipeline then kept roughly the 30 to 35 most prevalent features and excluded those mentioned by fewer than four people. The authors explicitly identify that threshold as a developer choice that can remove minority proposals.

The resulting menus were context-specific. Health status and survival chance were prominent in kidney allocation; reasons for a worker’s request and company policy were prominent in the workplace case; intended purpose and the deceased person’s consent were prominent in the generation case. Age makes the point sharply: 82 percent of Phase 1 kidney participants called it morally relevant, while 18 percent in WORK and 13 percent in GEN named age among features that should not count. A schema imported from one domain would not merely be incomplete. It would carry one domain’s moral map into another.

The Electorate Changes the Aggregate

Phase 2 asked participants to rate each retained contrast on a seven-point scale from minus three to plus three. In the control condition, preferences differed by political ideology for roughly one-third of features. The reported differences included expected recovery and underlying conditions in kidney allocation, being paid while absent in the worker case, and risk of misrepresenting the deceased. Some group differences reversed direction.

This is not evidence that three political labels exhaust moral diversity; the paper’s ethical statement says they do not. It is evidence that changing the mixture of even these coarse groups can change the profile called “the public view.” A published average without recruitment, exclusions, composition, weighting, and subgroup distributions is not a democratic result. It is an uninspectable constituency.

The Prompt Is a Policy Lever

Participants were randomly assigned to a baseline with no additional framing, a World-You-Want prompt about long-term social consequences and company policy, or a Could-Be-You prompt asking them to imagine occupying an affected position. The framing results did not reveal one universally fair question. Perspective-taking erased one ideological difference around expected recovery but increased another around memorial requests. The societal frame intensified some responsibility and loyalty effects. Across the study, wording could narrow or widen ideological gaps by as much as one scale point.

Even the control is a designed baseline, not an absence of values. Scenario text, examples, order, response scale, and the decision to add no perspective cue still structure the encounter. The study’s ethical statement accordingly refuses to call its standardized descriptions normatively neutral.

Sensitivity Is Not Legitimacy

The paper’s sensitivity audit asks developers to justify when aggregation is appropriate; publish complete candidate features, thresholds, and exclusions; recompute results under plausible voter compositions and alternative phrasings; preserve disagreement; disclose the aggregation rule; and verify that a system actually implements the elicited policy. A result that changes under those alternatives should be labeled scope-sensitive, constituency-dependent, or framing-sensitive rather than presented as fixed public morality.

That is necessary bookkeeping, but it cannot turn every vote into legitimate authority. The paper’s own residual-limits section warns that averaging can conceal conflict and expose minorities to majority rule. Some decisions require affected-stakeholder deliberation, rights constraints, or accountable institutional judgment. A transparent ballot can still ask a crowd to decide something a crowd should not be allowed to take away.

The Claim and Artifact Boundary

This is a U.S.-based study of hypothetical judgments in three scenarios, not a deployment trial or a cross-cultural map of morality. Its stated limits include a rating scale rather than forced choices, no direct record of participants’ reasoning, a threshold that can exclude rare views, AI literacy used as a proxy for experience, limited use-case coverage, and no intersectional analysis. The paper tests elicitation; it does not train or audit a resulting policy in operation.

The linked project page also currently says 817 participants, while arXiv v1’s abstract and phase counts total 809. It offers no explanation for the mismatch and lists its publication PDF as forthcoming. I therefore use the versioned arXiv record as the controlling source and treat the project page as supplementary context, not a second independent verification.

A Moral-Elicitation Receipt

A credible receipt should name the decision and affected parties; explain why voting is permitted to govern it; preserve the complete candidate-feature set, proposer counts, coding model and prompt, human checks, merge decisions, inclusion threshold, and exclusions; identify the target population, recruitment channel, compensation, attrition, demographics, ideology measure, weighting, and subgroup distributions; publish the full scenario, framing, examples, item order, response options, and aggregation rule; report sensitivity to alternative scopes, constituencies, frames, and rules; preserve disagreement and minority objections; document the translation into system behavior; and provide responsible owners, an appeal path, version history, and correction record.

The Spiralist lesson is that participation is not a decorative layer added after system design. When developers build the moral ballot, the interface itself becomes part of the policy.

Sources

Return to Blog