Blog · arXiv Analysis · Published: August 12, 2026 · Modified: August 12, 2026 · Last reviewed: August 12, 2026

The Persistent Instruction Becomes the Rationale Ledger

Kushal Chakrabarti studies why coding-agent context files accumulate instructions that later maintainers hesitate to remove.

A rationale ledger keeps each standing instruction tied to the failure, hypothesis, outcome, owner, and review decision that justified it.

The Paper

The paper is Kushal Chakrabarti's Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding, arXiv:2608.11095v1 [cs.AI], cross-listed in cs.LG and cs.SE. arXiv records submission on August 11, 2026. The 24-page version 1 PDF lists South Park Commons as the affiliation.

Chakrabarti calls the accumulation of retained but poorly understood instructions catastrophic remembering. The name is deliberately paired with catastrophic forgetting, but the object here is not a model's weights. It is a maintained text file such as CLAUDE.md, AGENTS.md, or copilot-instructions.md.

The Missing Maintenance Layer

This differs from the site's agent-config supply-chain essay, which asks where coding-agent rules came from and what authority they carry. Chakrabarti asks why a legitimate local rule remains after its reason has disappeared. Provenance identifies the source; rationale records the decision.

The distinction matters because a context file is both documentation and executable influence. A maintainer can add a defensive instruction after one failure in seconds. Months later, another maintainer may be unable to tell whether the rule prevents a regression, duplicates a newer rule, or survives only because deletion feels dangerous.

The Field Ratchet

The observational study follows 247,694 instruction lifetimes across 1,867 public GitHub repositories. It derives 1,801 multi-version context files and 299,440 version-to-version transitions. Across tracked file lifetimes, the paper reports mean instruction-count growth up to 226 percent; excluding mass rewrites, 19,267 commits add a mean net 4.9 instructions each.

Removal is lumpy. The paper classifies a commit that eliminates at least half of a file's live instructions as a rewrite and treats cross-file migration separately. Together, rewrites and migrations account for 77.3 percent of instruction deaths. In files aligned on their first rewrite, mean instruction count falls to 59.5 percent of its prior level and returns to 91.5 percent within ten commits.

Why Addition Wins

The proposed mechanism is imperfect recall. Instruction staleness predicts that old rules should become easier to delete. The paper instead reports a deletion log-hazard slope of -0.032 per commit, with a repository-stratified 95 percent interval from -0.047 to -0.019. A separate, exploratory model finds the age decline steeper in files touched by multiple human authors, while acknowledging that author count is only a proxy for turnover.

The paper's theory says rigorous pruning can require testing subsets of instructions because redundant rules make one-at-a-time removal inconclusive. Preserving the reason for an instruction can narrow that search. The exponential audit-cost argument is a model of worst-case counterfactual coverage, not a measured price for ordinary repository cleanup.

The Comment Intervention

The controlled experiment inverts IFEval so the minimum necessary instruction set is known. A model maintains prompts over repeated rounds while a harness supplies censored feedback. The executor sees only instructions; the next maintainer can also receive comments that record the triggering failure, the proposed cause, prior failed attempts, and observed outcome.

Across 552 maintenance histories at 15 steps, informative comments produce -5.8 percent mean excess size versus +60.4 percent without comments, at similar reported constraint satisfaction. At 51 steps, the corresponding figures are +1.4 percent and +211.3 percent. Comment-shaped noise does not reproduce the gain, and an ablation without outcomes performs poorly. The useful channel is not extra prose; it is preserved evidence about why a rule exists.

A WildIFEval extension seeds 16 unrelated instructions beside four or five human-written constraints. Under an arm-blind LLM judge, informative comments raise satisfaction from 50.4 to 62.0 percent over three maintenance rounds. A second judge reproduces the direction, but neither judge is ground truth.

The Governance Reading

Organizations should treat persistent agent instructions as revisable decisions, not sacred folklore. Every emergency rule needs an owner, a reason, evidence that it helped, and a condition for retirement. Otherwise yesterday's workaround becomes tomorrow's invisible policy, consuming context and interacting with newer rules.

This is also a labor issue. The person reviewing an old context file inherits risk without inheriting the event that created it. A rationale ledger transfers some of that missing institutional memory instead of asking each maintainer to reconstruct it from commit archaeology.

The Rationale Ledger

For each instruction, record its stable identifier, text, author, date, triggering task or incident, observed failure, causal hypothesis, alternatives tried, outcome, affected tests, scope, safety class, superseding rule, review date, and deletion decision. Keep rationale out of the executor's prompt when it is not operationally needed, but make it available to maintainers and auditors.

Pruning should produce its own receipt: proposed deletion, counterfactual tests, regressions checked, reviewer, rollback point, and post-change monitoring. The ledger is not permission to delete. It is evidence for deciding.

Limits and Deletion Risk

The field study is observational and depends on a fixed segmentation grammar, a 50 percent rewrite threshold, and a matcher validated by one author on 50 transitions. The controlled cover has only two or three instructions versus a field median of 39, uses one model as maintainer and executor, and tests mechanically verifiable English constraints. The WildIFEval arm seeds excess rather than observing it grow in real files.

Most importantly, the paper warns against automatic deletion. Its comment protocol sometimes empties prompts; on those worlds the uncommented arm has a higher satisfaction point estimate, but the paired interval includes zero. The author recommends keeping a person in the deletion path and excluding safety-relevant instructions until the method is tested on them. Comments can also become stale or misleading; rationale needs review just as instructions do.

Source Discipline

Primary sources were the arXiv abstract, metadata API, HTML, complete version 1 PDF, and arXiv source bundle. The PDF says derived tables and rebuilding code are released, but version 1 provides no external experimental-artifact URL and the source bundle contains the paper materials rather than that code. This essay reports the study without claiming an independent rerun.

Sources

Return to Blog