Blog · arXiv Analysis · Last reviewed July 10, 2026

The Compliance Agent Becomes the Evidence Boundary

The arXiv paper From Legacy Documentation to OSCAL shows a useful pattern for critical-infrastructure compliance: let agents assemble evidence, but make the first translation from prose into assets the inspected trust boundary.

Published: July 10, 2026 Modified: July 10, 2026 Last reviewed: July 10, 2026 Tags: arXiv, MCP, OSCAL, critical infrastructure, governance

The Passive Scan Problem

The paper is From Legacy Documentation to OSCAL: An MCP-Based Agent Pipeline for Threat-Informed Continuous Compliance in Critical Infrastructure, arXiv:2607.08288 [cs.CR], cross-listed in cs.AI. The arXiv record lists Lea Roxanne Muth and Marian Margraf as authors, records submission on July 9, 2026, and notes acceptance for the 2026 IEEE International Conference on Cyber Security and Resilience in Lisbon, August 3-5, 2026. The PDF metadata reports eight pages.

The paper starts from a practical constraint: operational-technology environments in critical infrastructure often cannot be actively scanned without availability risk. That leaves operators with a bad choice. They need current risk and compliance evidence, but the raw material may be old natural-language documentation, operator summaries, and passive exposure reports rather than a clean machine-readable asset inventory.

The authors' answer is not to let an LLM improvise a security assessment. It is to use Model Context Protocol tool calls as the retrieval boundary and OSCAL as the compliance artifact format. That makes this paper a useful companion to the Church's pages on tool-server trust boundaries, source-aware factuality, and agent action receipts.

What the Pipeline Does

The paper proposes a non-invasive, MCP-grounded multi-agent pipeline that converts natural-language system descriptions into a source-verified knowledge graph and audit-ready NIST OSCAL artifacts. It separates probabilistic agent phases from deterministic phases: the LLM helps extract and review, while MCP-backed data sources provide structured queries against threat-intelligence sources.

The authors describe fifteen active MCP servers grouped into six functional categories, including vulnerability foundations, risk filters, infrastructure visibility, threat taxonomies, resilience and defense, and lifecycle or supply-chain checks. The pipeline enriches a knowledge graph with entities, CVEs, weaknesses, attack patterns, ATT&CK techniques, and D3FEND countermeasures, then exports a System Security Plan and Security Assessment Report in OSCAL.

The crucial design choice is that the system does not treat vector similarity as enough for cyber evidence. CVE discovery, EPSS and KEV risk signals, taxonomy expansion, and D3FEND mapping are routed through structured tool calls. The governance claim is modest but important: bounded retrieval can make source failure more inspectable than free-form generation.

The WaterWork Test

The evaluation uses an evidence-based synthetic scenario called WaterWork: a regional water facility with high-pressure pumps, chlorine dosing, SCADA control, a Cisco ASA 5505 VPN gateway, Windows and SQL Server components, Siemens SIMATIC S7-1200 controllers, an Advantech WebAccess/SCADA workstation, and a vendor laptop. The authors manually construct ground truth with 292 CVEs across eight assets, 15 multi-stage attack paths, 16 ATT&CK Enterprise mappings, and 34 D3FEND countermeasure mappings.

Across five independent runs with identical input, the pipeline identifies eight entities and discovers 398 plus or minus 9 CVEs. Against the 292 ground-truth CVEs, it reports 0.90 recall and 0.74 precision, matching 263 ground-truth CVEs per run. Against D3FEND countermeasures, it reports 1.00 recall and 0.88 plus or minus 0.01 precision. The OSCAL SSP and SAR pass strict NIST OSCAL v1.1.2 JSON Schema validation in all five runs.

Bounded Retrieval Is Not Certainty

The strongest part of the paper is its refusal to call this solved. For deterministically sourced knowledge-graph nodes, the factual hallucination rate is 0 percent: the pipeline does not invent CVE identifiers or CVSS scores when those nodes come from validated sources. But the system still makes a semantic mistake at the front door.

Phase 0, the entity-extraction step, reaches 87.5 percent precision and 100 percent recall across the five runs. It also produces a 12.5 percent semantic hallucination rate: one broad "Windows" entity is extracted even though the described systems are specific Windows versions. That single false entity causes deterministic retrieval of real but irrelevant CVEs, and 30 of them survive into the final Security Assessment Report. The paper measures that as an 8.5 plus or minus 0.2 percent contextual false positive rate.

This is the real lesson. MCP grounding can move failure from fabricated identifiers into wrong scope. The bogus object is no longer a made-up CVE. It is a misread asset that pulls genuine evidence into the wrong context.

Governance Standard

A compliance agent should therefore be judged by where it concentrates review. The right test is not whether the final report sounds authoritative, or whether every artifact is schema-valid. The question is whether the pipeline identifies the exact point where human review breaks the cascade from a prose description to asset identity, vulnerability scope, risk score, taxonomy chain, control gap, and audit finding.

For operational technology, that review point is Phase 0. A human analyst should be able to inspect every extracted asset, vendor, version, topology role, criticality label, exposure label, and CPE assignment before deterministic retrieval begins. Once the asset list is wrong, every later tool can be perfectly obedient and still produce the wrong compliance queue.

Limits and Governance

The authors name the limits plainly. The evaluation relies on a single synthetic reference architecture, so cross-scenario variance remains unmeasured. The Critical Infrastructure Relevance Heuristic is transparent but needs further evaluation. The comparison to SSVC is indirect, not a dedicated expert study. The attack paths are software-vulnerability-centric and do not model credential theft, insider abuse, or social engineering.

The pipeline also depends on the timeliness and coverage of curated threat-intelligence sources. A source-backed system can still miss old advisories or lag new ones. That is why the paper's continuous-assessment posture matters: the same pipeline can be rerun as catalogs, advisory feeds, and OSCAL content change, but each run needs its own dated receipt.

The Receipt

A compliance-agent receipt should name the input documents, passive exposure report, model, prompt, extraction schema, extracted assets, rejected assets, CPE assignments, MCP server list, data-source versions, NVD and ICS-CERT query results, EPSS and KEV timestamps, CRH parameters, triage thresholds, taxonomy expansion rules, D3FEND mappings, OSCAL model version, JSON schema validator, generated SSP, generated SAR, manual review point, reviewer, changes made, rerun date, and residual unsupported threat classes.

The Spiralist reading is simple: machine-readable compliance is not the same thing as compliance. The record only becomes trustworthy when every transformation from prose to control evidence can be challenged.

Sources

Return to Blog