Latent Programming Horizons in Coding Agents
2026/07/06 by André Silva, Han Tu, Martin Monperrus · 10 voices
#cs.LG #cs.SE
paper · pdf
Abstract
A coding agent solving a software-engineering task spends dozens of steps reasoning, editing code, and running tests, yet little is known about what the underlying language model internally represents about the program it is working on. We show that the residual streams of language models under coding agents linearly encode properties of the evolving program: a logistic-regression probe on hidden states is able to decode whether the current code parses, passes its test suite, reduces the number of failing tests, and introduces regressions, reaching AUC up to 0.83 for correctness across two models and two benchmarks. Our second finding is more surprising: these representations run ahead of the agent's own edits. Probes trained to predict the outcome of future edits (before they are materialized and written on disk) achieve performance above chance up to roughly 25 steps in advance. We call this the agent's latent programming horizon. As a proof of external validity, we show that the probes transfer across benchmarks without retraining. Our positive results open calls for more research in mechanistic interpretability of coding agents.
Citations
Discussions
- Coding agents think ahead of time [hn, 96 points, 78 comments]
- Coding agents think ahead of time https://arxiv.org/abs/2607.05188 (https://news.ycombinator.com/item?id=48905764) [bsky, 1 points, 0 comments]
- [2607.05188] Latent Programming Horizons in Coding Agents — A clear look at how coding agents can plan and execute over longer “latent” horizons, with implications for reliability on multi-step tasks. [bsky, 1 points, 0 comments]
- 次世代のコーディングAIが登場。単なるコード補完を超え、先回りして課題を予測・解決する自律型エージェントが開発されています。開発効率を劇的に変えるこの技術は、プログラミングのあり方を根底から覆す可能性を秘めています。ソフトウェア開発の未来に注目です。 #AI #TechNews https://arxiv.org/abs/2607.05188 [bsky, 0 points, 0 comments]
- Coding agents think ahead of time #HackerNews https://arxiv.org/abs/2607.05188 [bsky, 0 points, 0 comments]
- Coding agents think ahead of time https://arxiv.org/abs/2607.05188 [bsky, 0 points, 0 comments]
- Coding agents think ahead of time https:// arxiv.org/abs/2607.05188 # arxiv [mastodon, 0 points, 0 comments]
- Coding agents think ahead of time [bsky, 0 points, 0 comments]
- New research on what coding agents represent internally, and how far ahead they plan in latent space before acting on a task. Submit to upvote on HN https://news.ycombinator.com/submitlink?u=https%3A% [bsky, 0 points, 0 comments]
- Coding agents think ahead of time https://arxiv.org/abs/2607.05188 (https://news.ycombinator.com/item?id=48905764) [bsky, 0 points, 0 comments]
Related