2025/08/03 by Zhilong Chen, Chengzong Zhao, Chen, Zhilong +19 · 2 citations
Computer Science · Decision Sciences · #Software Engineering Research #Scientific Computing and Data Management #Software Testing and Debugging Techniques
paper · pdf · doi:10.48550/arxiv.2508.01550
Training software engineering (SWE) LLMs is bottlenecked by expensive infrastructure, inefficient evaluation pipelines, scarce training data, and costly quality control. We present RepoForge, an autonomous, end-to-end pipeline that generates, evaluates, and trains SWE agents at scale. Our key contributions include: (1) RepoForge-8B-Agent, achieving 17.4% on SWE-Bench-Verified~\citepswebenchverified2024, establishing new state-of-the-art for ≤8B non-thinking LLMs; (2) 7,304 executable environments auto-generated from real GitHub commits with zero manual intervention; (3) 14× storage reduction (1.4GB → 102MB per instance) via intelligent dependency management and image pruning; (4) >70% faster evaluation using a Ray-powered~\citepray2018 distributed RepoForge harness; (5) 19,000× cheaper labeling through our automated SPICE~\citepspice2024 difficulty assessment technique. By unifying storage-efficient sandboxing, Ray-powered evaluation harness, automated data generation, SPICE-based labeling, and bubble-free RL scaffold, we demonstrate that even ≤8B models can reach new state-of-the-art performance on demanding benchmarks like SWE-Bench-Verified. Our approach addresses critical bottlenecks in SWE agent training: high storage costs of container-based evaluation, inefficient sequential reward pipelines, limited availability of high-quality training data, expensive manual labeling, and multi-turn RL pipeline bottlenecks.