2025/02/23 by Jonathan Light, Light, Jonathan, Wei Cheng +15 · 3 citations
Computer Science · #68T07 #68W40 #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #D.2.3 #F.2.2 #FOS: Computer and information sciences #I.2.6 #I.2.7 #I.2.8 #Machine Learning (cs.LG) #Mathematics, Computing, and Information Processing #Natural Language Processing Techniques #Software Engineering (cs.SE) #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2502.16706
openalex publication_date 2025/02/23 · openalex created_date 2025/10/09 · openalex updated_date 2026/07/28
Inference scaling methods for LLMs often rely on decomposing problems into steps (or groups of tokens), followed by sampling and selecting the best next steps. However, these steps and their sizes are often predetermined or manually designed based on domain knowledge. We propose dynamic decomposition, a method that adaptively and automatically partitions solution and reasoning traces into manageable steps during inference. By more effectively allocating compute -- particularly through subdividing challenging steps and prioritizing their sampling -- dynamic decomposition significantly improves inference efficiency. Experiments on benchmarks such as APPS, MATH, and LiveCodeBench demonstrate that dynamic decomposition outperforms static approaches, including token-level, sentence-level, and single-step decompositions, reducing the pass@10 error rate by 5.0%, 6.7%, and 10.5% respectively. These findings highlight the potential of dynamic decomposition to improve a wide range of inference scaling techniques.