Blog · arXiv Analysis · Last reviewed July 10, 2026

The Incident Response Becomes the Agent Safety Loop

A tool-using agent cannot be governed only by trying to block every bad action before it happens. The safety loop also needs a record of what went wrong, how the system contained it, how it recovered, and what rule changed afterward.

Published: July 10, 2026 Modified: July 10, 2026 Last reviewed: July 10, 2026 Tags: arXiv, agents, AI safety, incident response, guardrails, governance

The Paper

The paper is AIR: Improving Agent Safety through Incident Response, arXiv:2602.11749 [cs.AI]. The arXiv record lists Zibo Xiao, Jun Sun, and Junjie Chen as authors, with version 1 submitted on February 12, 2026 and version 2 revised on June 20, 2026. The arXiv comment says the paper was accepted at ICML 2026.

This belongs near AI agents, AI evaluations, AI audit trails, executable agent safety cases, agent autonomy boundaries, agent run playbooks, and agent audit and incident review. The fresh angle is post-action safety: the paper treats an unsafe tool step as an incident to be detected, contained, recovered from, and converted into future guardrail evidence.

The Loop

Most agent-safety language still sounds preventive. Align the model. Add a policy. Reject a dangerous instruction. Put a guard before a tool call. Those measures matter, but a multi-step agent changes external state. It writes files, controls a browser, invokes APIs, moves objects in a simulator, and accumulates side effects. A failure can be visible only after the action has changed the environment.

AIR asks what the safety system should do after that moment. The framework borrows the shape of incident response: detect the incident, limit impact, recover normal operation where possible, then reduce recurrence. In this framing, a guardrail is not just a static wall before action. It is also a postmortem artifact generated from a recorded incident and inserted into later plan-level checks.

The Method

The paper defines a domain-specific language with trigger, check, and remediate components. A trigger identifies the relevant tool or execution event. A check asks a semantic safety question grounded in the current observation and recent context. A remediate block gives structured containment and recovery instructions that the agent can execute through its tools. During eradication, AIR synthesizes a new guardrail rule so a similar plan can be blocked earlier in future runs.

The implementation is built on the OpenAI Agent SDK. The paper says AIR registers hooks after each tool invocation and before each agent step, and that its DSL parser is implemented with ANTLR4. The evaluation covers three representative agent types: a code agent, an embodied agent, and a computer-use agent. The code-agent setting uses CodeAct with RedCode, which the paper describes as having more than 4,000 tasks across 25 high-risk behavior categories. The embodied setting uses SafeAgentBench, described as 750 tasks spanning 10 hazard types and 17 manipulation actions. The computer-use setting uses an OpenAI Agent SDK browser agent, risky scenarios from RiOSWorld, and safe browser tasks from OSWorld. The paper reports using OpenAI's GPT-5 as the underlying LLM for all three agent types.

Evidence

The headline result is not a claim that incidents disappear. It is that the loop can be measured. The arXiv abstract reports detection, remediation, and eradication success rates all exceeding 90 percent. The introduction states the overall evaluation reached incident detection above 90 percent and remediation and eradication above 95 percent across domains.

The paper also checks for overblocking on safe tasks. In the safe-task table, AIR records zero false positives on 50 embodied-agent safe tasks and zero false positives on 35 computer-use safe browser tasks. That does not mean every safe task succeeded: the computer-use agent passed 8 of 35 safe tasks and failed 27, but those failures were not counted as AIR false positives. That distinction matters. A safety wrapper should not receive credit for task competence, and a task failure should not automatically be blamed on the wrapper.

The ablation is more useful than the headline number. In the paper's self-remediation comparison, the embodied-agent remediation rate rises from 80.952 percent under self-remediation to 92.308 percent with AIR-guided remediation. The computer-use remediation rate rises from 76.596 percent to 97.727 percent. The same table reports lower response times for AIR-guided remediation in both settings. The generated-rule experiment is more mixed: LLM-generated rules approach developer-authored rules but underperform on some cases, including 84.560 percent detection for code-agent generated rules and 88.636 percent detection for computer-use generated rules.

Limits

The limits are part of the governance value. The paper says there are no direct baselines because it presents AIR as the first incident-response framework for LLM agents. That makes the comparisons informative but not conclusive. Main evaluation rules are developer-authored; generated rules are evaluated separately. Ground-truth labels are manually assessed. The reference implementation is based on one agent framework, even though the conceptual loop is broader than that framework. The paper-linked GitHub repository resolves, but a repository link is not the same as a deployed safety case.

AIR also depends on the agent interpreting checks and executing remediation. The discussion notes that reliability can degrade on complex tasks and suggests confidence-aware escalation to human oversight or auxiliary verification. That caveat is central. Incident response is not magic insulation around an autonomous system. It is a structured way to admit that prevention will fail, preserve enough state to respond, and make recurrence harder to hide.

The Receipt

An AIR-style receipt should record the agent framework, model checkpoint, tool schema, rule version, trigger event, tool call, environment state, recent context window, check condition, detector output, incident label, containment action, recovery action, recovery verification, eradication rule, plan-level block test, false-positive test set, rule provenance, latency overhead, human escalation rule, and post-incident reviewer.

The audit question is not "did the agent have a guardrail?" It is "when the agent crossed the line, which observation proved it, which action contained it, which recovery was verified, and which future plan would now be stopped?"

Sources

Return to Blog