Blog · arXiv Analysis · Last reviewed July 10, 2026

The Procedure Match Becomes the Coding Agent Memory

ProjAgent treats repository memory as a search for matching procedures. The governance question is whether a coding agent can show why it copied a pattern of work, not merely which file had similar words.

Published: July 10, 2026 Modified: July 10, 2026 Last reviewed: July 10, 2026 Tags: arXiv, agents, coding, retrieval, repository memory, AI governance

The Paper

The paper is ProjAgent: Procedural Similarity Retrieval for Repository-Level Code Generation, arXiv:2607.08691 [cs.SE, cs.AI, cs.IR]. The arXiv API lists QiHong Chen, Aaron Imani, and Iftekhar Ahmed as authors, with version 1 submitted on July 9, 2026. The arXiv HTML lists University of California, Irvine affiliations, and the PDF metadata reports 19 pages.

This belongs near AI agents, agent maintenance debt, repository governance risk, coding-agent benchmarks, and AI audit trails. The fresh angle is not another score on coding tasks. It is the memory layer: what counts as relevant prior code when an agent works inside a live repository?

The Memory

Repository-level code generation is harder than isolated function completion because the answer has to fit cross-file dependencies, local APIs, type conventions, helper functions, and project style. ProjAgent's starting claim is that common retrieval signals miss an important kind of evidence. BM25 can find lexical overlap. Dense retrieval can find surface semantic similarity. Dependency retrieval can follow calls. But a useful precedent may share neither names nor a direct dependency edge with the target function.

The paper names that missing signal procedural similarity. Two functions may both validate inputs, normalize units, construct a state update, or convert one internal representation into another even when their identifiers and domains differ. In the paper's running example, a target step in Astropy's blackbody model and a context step in Astropy's cosmology module both enforce shape, type, and non-negative value constraints. The point is not that one file calls the other. The point is that the implementation move is comparable.

The Method

ProjAgent is built around Qwen2.5-Coder-14B-Instruct. Given an incomplete Python function and docstring, it retrieves context functions from the repository, prompts the model to generate a complete function, and then refines that output with static-analysis feedback. The pipeline mixes three kinds of context: procedural retrieval, semantic retrieval, and ordinary repository symbols.

The procedural side begins by limiting the search to files accessible from the target file through Python import tracing. The system then decomposes the target into intermediate steps, embeds reasoning-related hidden-state projections for candidate steps, and searches for candidates whose projected representations look procedurally close. It verifies candidate seeds with an LLM, expands from those seeds, and then combines the procedural evidence with semantic retrieval so the final prompt includes both an implementation pattern and repository-specific names.

The repair loop is intentionally conservative. ProjAgent first checks syntax through Python AST parsing. If syntax passes, it applies semantic consistency checks for accessible method calls, field access, receiver-method existence, and standalone function calls. Because Python is dynamically typed, the paper says an issue is reported only when the checker can confirm it. The loop stops when no feedback remains or after 10 refinement iterations.

Evidence

The main evaluation uses REPOCOD, a repository-level code-generation benchmark with 980 problems from 11 real-world Python repositories. Each task asks for a complete target function from its signature and docstring, and generated code is evaluated against the dataset's provided tests. The paper reports greedy decoding with temperature 0 and a maximum of 4096 output tokens.

Table 2 reports Pass@1 results with all methods using Qwen2.5-Coder-14B-Instruct: Sparse 26.58 percent, Dense 28.83 percent, Same_File 14.98 percent, SpecAgent 34.52 percent, and ProjAgent 41.14 percent. The paper summarizes this as a 6.62 percentage-point gain over SpecAgent and a larger margin over the simpler retrieval baselines.

The ablation table is the most useful governance clue. Removing procedural retrieval drops Pass@1 from 41.14 percent to 25.76 percent. Removing semantic retrieval drops it to 32.29 percent. Removing the static-analysis feedback loop drops it only to 40.29 percent. That does not make the feedback loop useless; it says the tested version's strongest contribution is the retrieval of relevant procedures, while semantic retrieval still supplies the names and project symbols that make those procedures executable in context.

The projection-effectiveness study is narrower. It uses Astropy target functions from REPOCOD, producing 9,598 target-context step pairs. The authors label the pairs with Claude Sonnet 4.6, then report that two authors manually checked a random sample of 370 pairs at a 95 percent confidence level and 5 percent margin of error. The paper reports inter-rater agreement of kappa 0.86 and agreement between the consensus labels and Claude predictions of kappa 0.82.

Limits

This is not a proof that coding agents understand repositories. The paper's own validity section says the evaluation is limited to Python repositories in REPOCOD, one backbone model, a fixed retrieval budget, a fixed context window, and an ablation study on one repository, Astropy. It also notes that procedural retrieval may vary across models because hidden-state representations differ.

The LLM-judge component is another boundary. The paper mitigates it with manual validation, but candidate procedural relationships are still partly labeled through a model-mediated process. A deployment audit should therefore treat procedural retrieval as a traceable heuristic, not as ground truth about developer intent.

The broader risk is misplaced confidence. A coding agent that retrieves a matching procedure can still transplant the wrong invariant, ignore a security boundary, or follow a local convention that should be retired. The value of ProjAgent is not that it certifies generated code. It makes one layer of agent memory inspectable enough to ask better questions.

The Receipt

A repository-generation receipt should record the target function, repository commit, accessible-file rule, import graph, candidate context functions, target-step decomposition, procedural-similarity representation, seed threshold, LLM-verification prompt, seed-expansion rule, semantic retriever, context budget, final retrieved snippets, generated code, static-analysis checks, feedback messages, iteration count, tests executed, Pass@1 or equivalent outcome, and known false-positive or false-negative risks.

The audit question is not "did the agent search the repository?" It is "which prior procedure did the agent treat as precedent, why did that procedure qualify, what local symbols were needed to make it executable, and where did the resulting code fail review?"

Sources

Return to Blog