Blog · arXiv Analysis · Last reviewed July 10, 2026

The Incident Report Becomes the Safety Memory

A July 2026 arXiv paper argues that post-deployment AI failures cannot teach the field unless they are defined, monitored, reported, classified, and analysed in comparable ways. The incident report is not paperwork after harm. It is the memory structure that determines whether the next system can learn.

Published: July 10, 2026 Modified: July 10, 2026 Last reviewed: July 10, 2026 Tags: arXiv, AI incidents, post-deployment monitoring, reporting, governance

The Paper

The paper is Harleen Kaur Sidhu, Rebecca Scholefield, Nour Annan, Kevin Hernandez, Isabel Nieh Hou, Abdulrahman Alshaikhi, Ze Shen Chin, and Rokas Gipiškis's Open Problems in AI Incident Governance, arXiv:2607.05163. The arXiv API lists version 1 as submitted on July 6, 2026, with primary category cs.CY and secondary category cs.AI, and its comment says it was accepted to the ICML 2026 Technical AI Governance Research workshop. The downloaded PDF is 21 pages.

The author affiliations in the paper include Independent, Sorbonne University, Rice University, Columbia University, AI Standards Lab, Oxford Martin AI Governance Initiative, and Vilnius University. This page belongs beside the site's work on AI incident reporting, agent incident response, vendor incident trust boundaries, and synthetic evidence. The fresh angle is incident reporting as shared safety memory.

What It Builds

The paper organizes AI incident governance as a pipeline. Definitions decide what counts as an incident. Taxonomies decide how incidents are grouped. Monitoring decides what signals are observed after deployment. Reporting decides what evidence reaches an authority or repository. Analysis decides whether the record can prevent recurrence.

The authors argue that existing frameworks cover pieces of this pipeline, but remain inconsistent across definitions, classification, monitoring, and reporting. The central operational gap is not only that incidents happen. It is that the field often lacks robust monitoring procedures and reporting templates that preserve enough information for later comparison.

The paper surveys corporate monitoring policies, proposes monitoring and reporting principles, operationalizes those principles as monitoring guidelines, and provides a reporting template in appendices. Its keywords are AI incident governance, incident reporting, post-deployment monitoring, and AI safety.

The Evaluation

This is not an empirical benchmark paper. Its evidence is a structured review of incident-governance frameworks from regulators and independent efforts. The definition section contrasts broad incident concepts, such as AIID-style harm or near-harm events, with narrower policy regimes that trigger duties around realized or serious harm. The paper identifies open questions about near misses, repeated failures, distributed harms, and whether definitions should be standardized.

The taxonomy section separates causal taxonomies from harm taxonomies. It discusses Goals, Methods, and Failures, CSET's AI Harm Framework, MIT AI Risk Repository taxonomies, and AIID's decision to host multiple taxonomies rather than force every incident into one view. The open problem is not simply "make one perfect taxonomy." It is whether category sets can remain useful as technology changes without becoming so broad that they stop explaining anything.

The monitoring section focuses on production systems, not pre-deployment testing. It treats incident monitoring as the systematic collection of operational and contextual data needed to detect incidents and understand root causes. The paper emphasizes a multi-actor ecosystem: providers and deployers have operational logs, but users, researchers, auditors, journalists, public databases, and market-surveillance bodies may see signals providers miss. It also cites an empirical finding that 38.3 percent of generative-AI cloud-service incidents in one production study were reported by humans rather than automated monitors.

The reporting section compares approaches from the EU AI Act, European Commission templates, OECD work, and CSET proposals. It asks what reports are for: immediate triage, causal investigation, pattern discovery, or regulatory accountability. That purpose determines whether a report needs structured categories, free-text narrative, timelines, impact estimates, system identifiers, deployment context, technical documents, or follow-up submissions.

Why It Matters

AI safety still leans heavily on the promise of pre-deployment evaluation: benchmark the model, red-team the obvious failures, document the release, and hope the safety case travels into production. The paper's premise is that the world will still produce failures the lab did not anticipate. Once that happens, governance depends on whether the event can be seen, named, reported, compared, and learned from.

This matters for agents, model APIs, public-sector systems, workplace tools, healthcare interfaces, and synthetic-media platforms. If an incident record lacks context windows, model version, tool permissions, deployment conditions, user reports, affected groups, and remediation status, then later analysis becomes rumor dressed as risk management.

What It Does Not Prove

The paper is agenda-setting, not a final standard. Its own impact statement describes the proposed monitoring guidelines and reporting template as starting points for harmonization, not final specifications. It does not prove that a single reporting template will work across all domains, jurisdictions, deployment architectures, and severity levels.

The harder problems remain open: near-miss definitions, reporting burden, privacy-preserving monitoring, voluntary public reporting, multi-system causality, alert fatigue, jurisdictional harmonization, and incident forecasting. The authors also disclose that Claude and Gemini were used for preliminary literature exploration and draft editing or review, with manual verification and author responsibility for final content.

Governance Reading

The Spiralist reading is that an institution remembers only what its forms can hold. If the reporting form recognizes death but not near miss, it forgets precursors. If the taxonomy records harm but not mechanism, it forgets causes. If monitoring captures outputs but not permissions, it forgets agency. If a voluntary report cannot be submitted by a technically ordinary person, the harmed public disappears from the evidence base.

Incident governance is therefore a memory politics problem. The field is not only deciding how to document failures. It is deciding which failures become comparable enough to matter.

The Receipt

An AI incident safety-memory receipt should record the incident ID, report stage, reporter role, affected people or systems, observed harm and near-miss status, start time, detection time, end or remediation time, uncertainty about dates, model and product identifiers, deployment context, inputs and outputs preserved for review, tool calls, permissions, human oversight, safeguards, alerts, user reports, suspected contributing factors, severity assessment, remediation, recurrence checks, and privacy limits on retained data.

The practical rule: a failure that is not reportable, searchable, comparable, and reviewable has not entered governance. It is only an anecdote waiting to happen again.

Sources

Return to Blog