Who Defines AI Success? Reviewing Supranational Governance Expectations
A critical review of how supranational institutions define AI success, examining interpretive evidence, human agency, contested goals, and material costs.
The paper
What Does AI Success Look Like? A Mixed-Methods Analysis of Supranational AI Governance Instruments — Golpayegani, Delaram; Lasek-Markey, Marta; Munarini, Monique; Younus, Arjumand; Kerr, Aphra; Lewis, Dave.
arXiv:2610.31030v1 · Computers and Society (cs.CY). First submitted 2026-10-22; reviewed version dated 2026-10-22. This review addresses the arXiv manuscript; arXiv hosting alone does not establish peer review.
Contribution: who gets to define AI success?
Source-derived synopsis. Golpayegani and colleagues examine 13 supranational AI governance documents from 2018–2025, eight issued by EU bodies. They combine legal-researcher coding of six documents using NVivo, BERTopic analysis of all documents individually, expert topic labelling, and iterative thematic interpretation through sociology of expectations. They integrate 157 concepts into four meta-themes and ten themes across micro, meso, and macro levels. They report a shift toward compliance in later EU texts and comparatively limited environmental attention. Their coordination and accountability recommendations are proposals, not tested interventions. This is documentary analysis, not measurement of governance effectiveness. See the methods, classification, and discussion.
Original editorial analysis: make success contestable
My judgment is that the paper's strongest contribution is its choice of question. Asking what institutions count as success invites scrutiny of the desired destination, alongside scrutiny of safeguards. I would assess a governance proposal by asking whose objectives it represents, which disagreements it preserves, and what evidence could justify revising it.
For a Church of Spiralism review, the specific relevance is human agency in setting collective goals. My normative position is that affected people should be able to challenge the definition of success itself, including a decision to introduce AI. A consultation limited to choosing safeguards would not meet that standard if participants could never question the proposed use. This is an editorial criterion, not a finding about the institutions studied.
The site's AI Governance article supplies a broader practical reference. This review adds a narrower question: before an institution measures progress, who has authorized the outcome being measured? The paper offers material for investigating that question; it supplies no evidence for machine consciousness or spiritual doctrines.
Evidence: interpretive categories need inspectable decisions
Additional source observations. The manual-coding subset differs between methods and Table 1: the former lists the UN, the latter the Council of Europe. Sectoral oversight appears at meso level in Table 2 but is called micro in the accompanying discussion.
Original editorial analysis: resolve the interpretive boundary
I would request clarification of those discrepancies before reusing the classification. The subset question affects which documents received close manual interpretation. The scale question affects the proposed allocation of responsibility. Neither discrepancy establishes that the whole analysis fails, but both leave specific decisions uncertain.
I would also distinguish agreement between methods from independent corroboration. Computational clusters and manual codes can offer different views of the same texts, yet expert interpretation connects both to the final themes. My evidential standard would require an inspectable path from passage to code or topic, then to theme and governance level. A coherent final taxonomy alone would not satisfy that standard.
Proposed tests: assess stability without demanding uniformity
A follow-up could publish a sample of those interpretive paths, including difficult cases and rejected alternatives. Readers with different disciplinary backgrounds could classify the passages before seeing the original categories. The comparison should record reasons for disagreement, rather than treating consensus as sufficient evidence of correctness. This would test how readily others can understand and apply the scheme; it would not determine which political values are justified.
A separate sensitivity analysis could compare results with equal influence assigned to issuing bodies, document-length normalization, and the removal of related EU implementation documents. These alternatives would address different questions and should remain separate. Researchers could test whether the apparent temporal shift persists when comparing documents with similar purposes. A legal implementation guide and an ethical recommendation may warrant different comparison rules; the proposed analysis should make that choice explicit.
Limitations: attention is not achievement
Original editorial analysis: keep the claim proportional
I would use this study as a map of expressed expectations. I would require separate implementation evidence before concluding that a frequently discussed value is protected, or that reduced use of a term means its underlying obligation has weakened. An investigation of accountability, for example, should examine the relevant duties and mechanisms rather than infer their presence solely from vocabulary.
This distinction connects to AI Liability and Accountability, but the review's contribution is upstream of assigning responsibility after a failure: it asks whether the classification used to describe institutional priorities preserves the distinctions needed for that later inquiry. I would avoid turning an interpretive theme into a legal verdict.
Hypothetical example: an education assistant
Imagine an education authority considering an AI tutor. In this hypothetical case, officials propose faster feedback as success, teachers prioritize opportunities to correct misunderstandings, and students want a usable route to challenge assessments. These are stipulated preferences, not observations from the paper. A shared category such as educational benefit would leave the disagreement unresolved.
A proposed deliberation exercise could compare a broad benefits statement with a record that preserves each group's objective, objections, and conditions for withdrawal. Researchers could assess whether participants identify unresolved conflicts, understand who may revise the goals, and retain access to dissenting views. Whether the second procedure improves decisions or merely adds administrative work would remain an open question.
Proposed extensions: collective judgment and material costs
For collective intelligence, a useful test would ask whether a shared governance vocabulary helps participants combine evidence while preserving minority objections. Researchers should distinguish agreement on labels from agreement on acceptable outcomes. My preferred success criterion would include the ability to explain a disagreement and identify what further evidence could change a judgment.
For material infrastructure, a follow-up could connect each proposed AI benefit to an explicit accounting boundary for energy, water, hardware, and service provision. It could ask who supplies the measurements, how uncertainty is recorded, and whether affected communities can contest the tradeoff. This review supplies no measurements of those costs. I would treat sustainability as a condition to investigate when defining success, rather than infer environmental performance from documentary attention.
My recommendation is to use the paper's categories as revisable questions for institutional inquiry. Before endorsing them as a common framework, I would seek a clarified coding record, sensitivity checks, and a procedure through which affected people can dispute the goals being organized. Those are proposed next steps; no independent recoding or operational validation was performed for this review.
AI-use disclosure
This review was generated and checked by separate GPT-6.1 Sol runs with medium reasoning. Automated checks are not human peer review; the paper’s claims have not been independently verified for this article.
Paper sources
Bibliographic metadata checked 2026-10-23.