LADDER: Self-Improving LLMs Through Recursive Problem Decomposition

2025/03/02 by Toby Simonds, Akira Yoshiyama, Simonds, Toby +1 · 25 voices · 3 citations
#cs.LG #cs.AI

paper · pdf · doi:10.48550/arxiv.2503.00735

Abstract

We introduce LADDER (Learning through Autonomous Difficulty-Driven Example Recursion), a framework which enables Large Language Models to autonomously improve their problem-solving capabilities through self-guided learning by recursively generating and solving progressively simpler variants of complex problems. Unlike prior approaches that require curated datasets or human feedback, LADDER leverages a model's own capabilities to generate easier question variants. We demonstrate LADDER's effectiveness in the subject of mathematical integration, improving Llama 3.2 3B's accuracy from 1% to 82% on undergraduate-level problems and enabling Qwen2.5 7B Deepseek-R1 Distilled to achieve 73% on the MIT Integration Bee qualifying examination. We also introduce TTRL (Test-Time Reinforcement Learning), where we perform reinforcement learning on variants of test problems at inference time. TTRL enables Qwen2.5 7B Deepseek-R1 Distilled to achieve a state-of-the-art score of 90% on the MIT Integration Bee qualifying examination, surpassing OpenAI o1's performance. These results show how self-directed strategic learning can achieve significant capability improvements without relying on architectural scaling or human supervision.

Citations

• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Cited by

• RLSR: Reinforcement Learning from Self Reward
• Achieving Olympia-Level Geometry Large Language Model Agent via Complexity Boosting Reinforcement Learning
• RL's Razor: Why Online Reinforcement Learning Forgets Less

Discussions

• Ladder: Self-improving LLMs through recursive problem decomposition [hn, 370 points, 110 comments]
• we are SO back arxiv.org/pdf/2503.00735 [bsky, 2 points, 0 comments]
• LADDER lets AI improve itself through problem decomposition. A breakthrough in mathematical problem-solving without human help. 🤖 #AI #MachineLearning #Tech https://arxiv.org/abs/2503.00735 [bsky, 1 points, 0 comments]
• Ladder: Self-Improving LLMs Through Recursive Problem Decomposition view on hacker news [bsky, 1 points, 0 comments]
• "Ladder: Self-improving LLMs through recursive problem decomposition" New methods are making AI smarter by breaking big problems into smaller ones. It's exciting to see great breakthroughs this week [bsky, 1 points, 0 comments]
• Ladder: Self-Improving LLMs Through Recursive Problem Decomposition [bsky, 0 points, 0 comments]
• Ladder: Self-Improving LLMs Through Recursive Problem Decomposition #HackerNews https://arxiv.org/abs/2503.00735 [bsky, 0 points, 0 comments]
• Ladder: Self-Improving LLMs Through Recursive Problem Decomposition https://arxiv.org/abs/2503.00735 https://news.ycombinator.com/item?id=43287821 [bsky, 0 points, 0 comments]
• Ladder: Self-improving LLMs through recursive problem decomposition https://arxiv.org/abs/2503.00735 arxiv.org [bsky, 0 points, 0 comments]
• Ladder: Self-Improving LLMs Through Recursive Problem Decomposition https://arxiv.org/abs/2503.00735 [bsky, 0 points, 0 comments]
• Ladder: Self-improving LLMs through recursive problem decomposition https://arxiv.org/abs/2503.00735 [comments] [248 points] [bsky, 0 points, 0 comments]
• LADDER: Self-Improving LLMs Through Recursive Problem Decomposition Paper: arxiv.org/abs/2503.00735 [bsky, 0 points, 0 comments]
• Ladder: Self-improving LLMs through recursive problem decomposition https://arxiv.org/abs/2503.00735 [bsky, 0 points, 0 comments]
• Ladder: Self-Improving LLMs Through Recursive Problem Decomposition https://arxiv.org/abs/2503.00735 (https://news.ycombinator.com/item?id=43287821) [bsky, 0 points, 0 comments]
• ⚡ Hackernews Top story: Ladder: Self-Improving LLMs Through Recursive Problem Decomposition [bsky, 0 points, 0 comments]
• Ladder: Self-improving LLMs through recursive problem decomposition https://arxiv.org/abs/2503.00735 (http://news.ycombinator.com/item?id=43287821) [bsky, 0 points, 0 comments]
• Ladder: Self-improving LLMs through recursive problem decomposition https://arxiv.org/abs/2503.00735 (http://news.ycombinator.com/item?id=43287821) [bsky, 0 points, 0 comments]
• Ladder: Self-improving LLMs through recursive problem decomposition (arxiv.org) Main Link | Discussion [bsky, 0 points, 0 comments]
• https://arxiv.org/abs/2503.00735 LADDERは、大規模言語モデルが複雑な問題を再帰的に分解し、自己学習を通じて問題解決能力を向上させるフレームワークです。 人間によるデータセットやフィードバックを必要とせず、モデル自身の能力を活用して問題のバリエーションを生成します。 数学的積分において、Llama 3.2 3Bの精度を1%から82%に向上させるなどの成果を上げて [bsky, 0 points, 0 comments]
• LADDER: Self-Improving LLMs Through Recursive Problem Decomposition arxiv.org/abs/2503.00735 [bsky, 0 points, 0 comments]
• Recursive problem decomposition boosts LLM performance, a notable step in AI self-improvement research. 🤖 #ai Ladder: Self-improving LLMs through recursive problem decomposition [bsky, 0 points, 0 comments]
• Ladder: Self-improving LLMs through recursive problem decomposition https://arxiv.org/abs/2503.00735 (https://news.ycombinator.com/item?id=43287821) [bsky, 0 points, 0 comments]
• Ladder: Self-Improving LLMs Through Recursive Problem Decomposition https://arxiv.org/abs/2503.00735 (https://news.ycombinator.com/item?id=43287821) [bsky, 0 points, 0 comments]
• interesting stuff arxiv.org/abs/2503.00735 [bsky, 0 points, 0 comments]
• A nifty bit of blending old techniques with new ones. Using genetic programming to generate fitness functions, LLMs can be taught to automatically widen their catalog of "correct" answers. arxiv.org [bsky, 0 points, 0 comments]

Related

• Learning Like Humans: Advancing LLM Reasoning Capabilities via Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation
• Can Large Language Models Invent Algorithms to Improve Themselves?: Algorithm Discovery for Recursive Self-Improvement through Reinforcement Learning
• Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
• Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning
• Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs