The Culture Meter Becomes the Apparatus
When a language model measures culture, the instrument is not outside the scene. It is one of the things that has to be audited.
The Paper
The paper is Kent K. Chang's Language Models as Measurement Apparatus for Culture, arXiv:2607.02459 [cs.CL]. The arXiv record lists version 1 as submitted on July 2, 2026, and the PDF is 14 pages. The author affiliation on the paper is the School of Information, University of California, Berkeley. The arXiv metadata also links DOI 10.18653/v1/2026.bigpicture-main.11 and says the work was accepted to the Big Picture workshop co-located with ACL 2026. The ACL Anthology page lists it in Proceedings of The Big Picture v2: Crafting a Research Narrative, pages 131-143, San Diego, CA, USA.
The paper's question is not whether language models can classify cultural artifacts. Its sharper question is what kind of measurement is happening when the measuring instrument has already absorbed cultural material, and when the categories chosen by the researcher decide what can become visible.
The Cut
Chang adapts Karen Barad's term agential cut for cultural analytics. In this constrained methodological use, the cut is the boundary a research apparatus draws between phenomenon and instrument. A model architecture, training corpus, label inventory, annotation rule, input representation, and evaluation metric do not merely record culture from a distance. Together, they decide whether a scene becomes a reply graph, a role-label problem, a relationship classification task, a deviation from a norm, or something that falls outside the measurable frame.
That is the governance point. A dashboard that says a model measured gender, agency, role, stereotype, subversion, sentiment, ideology, or cultural alignment is already downstream of a theory of what counts. The number is not only a result. It is a record of a boundary choice.
The Case Studies
The arXiv version develops three case studies on television and film dialogue. The first is structure: conversation disentanglement turns interleaved multi-party dialogue into directed reply-to links. The second is interaction: conversational role attribution maps a scene into speaker, addressee, and side-participant labels, then supports claims about gendered participation. The third is deviation: stereotypic relation extraction treats departures from learned relationship expectations as the measurement object.
Those examples matter because they are not interchangeable views of one raw truth. A reply graph can make initiation and response visible. A Goffman-inspired role taxonomy can make listening positions visible. A stereotypic-relation model can make certain forms of deviation visible. Each apparatus also leaves something out. A cultural claim should therefore travel with the apparatus that made it possible.
The Instrument Has Memory
The paper then turns the measuring instrument back on itself. In the erasure analysis, replacing character names with anonymous identifiers sharply reduces performance in one multimodal conversation-structure study: speaker recognition drops from 78.6 to 13.7, and addressee recognition drops from 68.1 to 15.7. Chang's interpretation is not that names are bad features in the abstract. It is that the apparatus was partly drawing on cultural memory tied to recognizable characters.
The attunement section examines Restoration comedy from 1660-1700. The paper describes a toy experiment using 109 plays and 1,283 character episodes from the Chadwyck-Healey English Drama collection, with archetype labels developed through criticism, hand annotation, prompt refinement, and Gemini 2.5 Flash scaling. It reports Cohen's kappa of 0.71 against the author's annotation of five plays, and one epoch of continued pre-training on roughly 174,000 lines of in-domain dialogue.
The arXiv version also adds an agency section about distributing measurement across an agentic workflow of several model backbones. This is not a claim that the models have minds or cultural authority. It is a warning that the measurement apparatus can become a workflow, with multiple memories, prompts, and intermediate decisions between the artifact and the final number.
The Culture-Measurement Receipt
A culture-measurement receipt should include the artifact corpus, licensing or access limits, train and test boundaries, model names and versions, prior-exposure assumptions, prompts, label inventories, annotation rules, adjudication process, perturbation tests, task framing, metrics, uncertainty, excluded categories, theoretical commitments, and the exact claim the number is allowed to support.
Without that receipt, "the model found a cultural pattern" is too loose. It might mean the model inferred a relation from dialogue, recognized names from pretraining, followed an annotation scheme, reproduced the researcher's categories, or exposed a mismatch between a stereotype and a performance. The output may still be useful, but only if the audience can see the apparatus.
Limits
This is a big-picture and theory-building paper, not a universal benchmark for cultural competence. Its examples draw heavily on scripted film, television, and drama, and several empirical pieces are inherited from or connected to prior studies. The Restoration section is explicitly framed as a toy experiment. The strongest practical lesson is modest: culture cannot be measured by pretending the model is a neutral caliper. The claim, the corpus, the categories, and the model have to be reviewed together.
Sources
- Kent K. Chang, Language Models as Measurement Apparatus for Culture, arXiv:2607.02459 [cs.CL].
- arXiv HTML for Language Models as Measurement Apparatus for Culture, checked for title-page metadata, abstract, case-study structure, erasure, attunement, agency, appendix, and conclusion.
- arXiv PDF for Language Models as Measurement Apparatus for Culture, checked for page count, author affiliation, submission line, and body text.
- ACL Anthology entry 2026.bigpicture-main.11, checked for DOI, pages 131-143, proceedings title, publisher, month, year, and address.
- The Big Picture Workshop 2026 page, checked for ACL 2026 co-location, San Diego date, and the Kent K. Chang spotlight listing.