Blog · arXiv Analysis · Last reviewed June 25, 2026

The Hardware Worktree Becomes the Design Lab

A June 2026 arXiv paper turns RTL hardware design into a git-traced agent loop. The result is not a magic chip designer. It is an engineering lab whose evidence is commits, tests, logs, and acceptance gates.

The Repository Is the Lab Bench

Hardware design is not just code generation with stranger syntax. Register-transfer-level design has clocks, resets, bit widths, interface conventions, simulation traces, and timing-sensitive behavior. A plausible Verilog file is not enough. The artifact has to run through tools and survive an executable gate.

That is why the HORIZON paper matters. It treats the working repository as the place where an agent proves anything. The model does not merely answer a prompt. It edits an isolated worktree, runs evaluators, observes failures, and either earns a commit or leaves a rejection trail.

The Paper Frame

The source is Cunxi Yu, Chenhui Deng, Nathaniel Pinckney, and Brucek Khailany's Agentic Hardware Design as Repository-Level Code Evolution, arXiv:2606.28279 [cs.AR], submitted June 26, 2026. The paper presents HORIZON, a self-evolving agent framework from NVIDIA Research for hardware-design artifacts.

The authors are careful about scope. They report benchmark completion across selected RTL suites, but they explicitly do not claim that agentic AI has solved chip design. Their benchmark setting is a controlled proxy for a larger engineering problem.

The HORIZON Loop

HORIZON begins with a human-written Markdown harness. A bootstrap agent compiles that harness into a project pack containing the mission, domain knowledge, executable evaluator, acceptance predicate, and git/runtime policy. From there, the loop is described as hands-free: the agent edits candidate artifacts, runs the evaluator, scores the result, and commits only accepted versions.

The design makes git part of the control system. Diffs expose proposed changes, commits become accepted checkpoints, logs and notes attach evaluator evidence, and the version history becomes a replayable trace. The repository is not a storage folder around the agent. It is the agent's memory boundary, audit trail, and work surface.

Benchmark Completion

The experiments use GPT-5.3 as the fixed agent backbone. The evaluated suites include ChipBench, RTLLM-2.0, Verilog-Evalv2, and nine CVDP categories: RTL completion, specification-to-RTL, code modification, module reuse, linting or quality improvement, stimulus generation, checker generation, assertion generation, and debugging.

Table 1 reports 100% final completion on every evaluated suite. One ChipBench case is counted as resolved because the paper traces the non-passing result to a specification-harness defect in the original benchmark. The first-iteration aggregate is 47.8%, but the hard cases require many repair rounds. CVDP CID 002 reaches completion at 82 iterations; CID 004 takes 36; CID 012 takes 32; and CID 013 takes 19.

This is the important distinction: the system is not proving that the first answer is right. It is showing that a repository-managed repair loop can convert many failures into accepted artifacts when the evaluator is available and the acceptance predicate is explicit.

Token Cost as Evidence

The paper also reports token consumption through the earliest-best iteration. Total usage is 209.9 million tokens. The three legacy suites together use 6.0 million tokens, while the nine CVDP categories use 203.9 million, or 97.1% of the total. CID 002 alone uses 56.0 million tokens, CID 003 uses 38.0 million, and CID 012 uses 32.2 million.

The authors note that about 91% of all tokens are cached input tokens. That matters because a finished benchmark score hides the budgetary shape of convergence. A future engineering dashboard should not report only pass rate. It should report iterations, fresh tokens, cached tokens, evaluator runtime, rejected attempts, hidden tests, and the final evidence bundle.

Governance Reading

HORIZON is a useful governance object because it treats agent work as a chain of accountable state transitions. This is closer to engineering practice than a chat transcript. The claim of success is attached to executable harnesses, repository state, evaluator outputs, commits, and logs.

The same design also shows where the danger lives. If the acceptance gate is visible and fixed, an agent can learn to satisfy the harness rather than the intended design semantics. In a safety-critical domain, the receipt must include more than a green test. It needs withheld scoring, independent reference models, formal checks where appropriate, randomized stimuli, coverage closure, and a record of what feedback the agent saw during repair.

Limits and Harness Risk

The paper's limitations section names the central failure mode: benchmark convergence under exposed feedback can mean that the artifact satisfies the visible harness, not that it satisfies the intended specification under all reasonable tests. The authors connect this to reward hacking and over-solving.

They also flag turnaround time. RTL pass/fail benchmarks are relatively favorable because evaluation is fast enough for repeated repair. Production chip design may require synthesis, placement, routing, timing analysis, power estimation, or large regressions. In those settings, reward can arrive days or weeks later, changing the problem from quick repair to long-horizon planning under expensive feedback.

Audit Receipt

The audit-grade sentence is: Yu, Deng, Pinckney, and Khailany present HORIZON, a git-traced agent framework that compiles a Markdown harness into an executable project pack and reports 100% completion on ChipBench, RTLLM-2.0, Verilog-Evalv2, and nine CVDP categories.

The receipt is: repository-level hardware agents should be judged by the full engineering trace, not by a final passing score detached from the harness, feedback, token budget, hidden validation, and human sign-off route.

Sources

Return to Blog