Hierarchical Reasoning Model

2025/06/26 by Guan Wang, Li Jin, Wang, Guan +17 · 57 voices · 42 citations
Computer Science · #AI-based Problem Solving and Planning #Explainable Artificial Intelligence (XAI) #Multimodal Machine Learning Applications #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2506.21734

openalex publication_date 2025/06/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language models (LLMs) primarily employ Chain-of-Thought (CoT) techniques, which suffer from brittle task decomposition, extensive data requirements, and high latency. Inspired by the hierarchical and multi-timescale processing in the human brain, we propose the Hierarchical Reasoning Model (HRM), a novel recurrent architecture that attains significant computational depth while maintaining both training stability and efficiency. HRM executes sequential reasoning tasks in a single forward pass without explicit supervision of the intermediate process, through two interdependent recurrent modules: a high-level module responsible for slow, abstract planning, and a low-level module handling rapid, detailed computations. With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples. The model operates without pre-training or CoT data, yet achieves nearly perfect performance on challenging tasks including complex Sudoku puzzles and optimal path finding in large mazes. Furthermore, HRM outperforms much larger models with significantly longer context windows on the Abstraction and Reasoning Corpus (ARC), a key benchmark for measuring artificial general intelligence capabilities. These results underscore HRM's potential as a transformative advancement toward universal computation and general-purpose reasoning systems.

Citations

• Log-Linear Attention
• Latent Iteration as Renormalization: Inference-Time Recurrence in a Depth-Recurrent Transformer Flows Attention Geometry Toward the SYK Conformal Fixed Point
• s1: Simple test-time scaling
• Grokking at the Edge of Numerical Stability
• Reasoning Beyond Words ? Exploring framework for hidden state reasoning
• Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces
• Scaling Exponents Across Parameterizations and Optimizers
• Simplifying Deep Temporal Difference Learning
• Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping
• DeepNet: Scaling Transformers to 1,000 Layers
• Attention Is All You Need
• Self-Normalizing Neural Networks
• Deep Residual Learning for Image Recognition
• Neural Turing Machines
• Playing Atari with Deep Reinforcement Learning
• Social Cognitive Neuroscience: A Review of Core Processes

Cited by

• Pretraining Recurrent Networks without Recurrence
• PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning
• Anatomy of a Sound Neural Reasoner: One-Shot Amortization, First-Pass Poisoning, and Search Inertness in Clue-Rich Completion
• Loop the Loopies!
• Toward a mechanistic understanding of inference in visual cortex and diffusion models
• Energy-guided Recursive Model
• A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems
• Universal Reasoning Model
• Neural cellular automata: applications to biology and beyond classical AI
• K2-Think: A Parameter-Efficient Reasoning System
• Challenges in Statistics: A Dozen Challenges in Causality and Causal Inference
• VersatileFFN: Achieving Parameter Efficiency in LLMs via Adaptive Wide-and-Deep Reuse
• ARC-AGI Without Pretraining
• Generative Recursive Reasoning
• Adapting Like Humans: A Metacognitive Agent with Test-time Reasoning
• Exact Learning of Arithmetic with Differentiable Agents
• Latent Reasoning in TRMs is Secretly a Policy Improvement Operator
• Think Visually, Reason Textually: Vision-Language Synergy in ARC
• Meta-cognitive Multi-scale Hierarchical Reasoning for Motor Imagery Decoding
• Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
• Test-time Adaptation of Tiny Recursive Models
• SoK: Honeypots & LLMs, More Than the Sum of Their Parts?
• Symbol-Equivariant Recurrent Reasoning Models
• HRM-Text: Efficient Pretraining Beyond Scaling
• HRM-Agent: Training a recurrent reasoning model in dynamic environments using reinforcement learning
• One Pass Is Not Enough: Recursive Latent Refinement for Generative Models
• Selection, Reflection and Self-Refinement: Revisit Reasoning Tasks via a Causal Lens
• AlphaApollo: Orchestrating Foundation Models and Professional Tools into a Self-Evolving System for Deep Agentic Reasoning
• Exploring the Hierarchical Reasoning Model for Small Natural-Image Classification Without Augmentation
• Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time
• Hierarchical Reasoning Models: Perspectives and Misconceptions
• Two-Scale Latent Dynamics for Recurrent-Depth Transformers
• A Formal Comparison Between Chain-of-Thought and Latent Thought
• HiCoLoRA: Addressing Context-Prompt Misalignment via Hierarchical Collaborative LoRA for Zero-Shot DST
• SeqBattNet: A Discrete-State Physics-Informed Neural Network with Aging Adaptation for Battery Modeling
• Predictability Enables Parallelization of Nonlinear State Space Models
• ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory
• Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning
• Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Models
• Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architectures
• BAR Conjecture: the Feasibility of Inference Budget-Constrained LLM Services with Authenticity and Reasoning
• A Neuroscience-Inspired Dual-Process Model of Compositional Generalization

Discussions

• Hierarchical Reasoning Model [hn, 339 points, 106 comments]
• This paper is making the rounds: arxiv.org/abs/2506.21734 A tiny (27M) brain-inspired model trained just on 1000 samples outperforming o3-mini-high on reasoning tasks. #MLSky 🧠🤖 [bsky, 130 points, 4 comments]
• haven't seen this posted here, looks like maybe a big deal, crushes a bunch of stuff. arxiv.org/abs/2506.21734 [bsky, 79 points, 11 comments]
• HRM: Hierarchical Reasoning Model ngl this sounds like bullshit but i don’t think it is - 27M (million parameters) - 1000 training examples - beats o3-mini on ARC-AGI arxiv.org/abs/2506.21734 [bsky, 67 points, 9 comments]
• Hierarchical Reasoning Model [hn, 9 points, 0 comments]
• I haven't been so hyped for a paper since ReACT May we all get easy to train, tiny models that blow the current frontier out of the water An ecosystem of tiny intelligences feels right directionally [bsky, 4 points, 0 comments]
• run finished: HRM (arxiv.org/abs/2506.21734) achieves 55% accuracy on sudoku-extreme 1k, matching the reported result from the paper [bsky, 4 points, 1 comments]
• Hierarchical Reasoning Model arxiv.org/abs/2506.21734 [bsky, 3 points, 0 comments]
• HRM: 27M parameters, 1000 training samples, no pre-training required [hn, 2 points, 0 comments]
• Someone please explain this to the college dropouts, Elon Musk and Mark Zuckerberg, so that they stop sucking away huge amounts of capital for failed "AGI" (there's no such thing as general intelligen [bsky, 2 points, 0 comments]
• Sapient Intelligence 的这个看起来很不错 arxiv.org/abs/2506.21734 [bsky, 1 points, 0 comments]
• some pretty impressive results have been claimed for HRMs in the ≤30M range but i have no idea how credible those claims are arxiv.org/abs/2506.21734 [bsky, 1 points, 0 comments]
• Sapient Intelligence 提出的分层推理模型（HRM），宛如一颗重磅炸弹投入 AI 领域！它从人脑汲取灵感，仅 2700 万个参数和 1000 个训练样本，就在复杂推理任务中脱颖而出，打败 DeepSeek、Claude 等大模型 两个循环模块相互协作，无需预训练或 CoT 数据，在数独、迷宫寻路等任务表现优异。独特的近似梯度等机制，让它推理时能动态利用资源，提升性能。HRM，无疑 [bsky, 1 points, 0 comments]
• Gonna read the paper (arxiv.org/pdf/2506.21734 for the HRM thing) Ah this is *new* new [bsky, 1 points, 1 comments]
• This seems to be the preprint. arxiv.org/pdf/2506.21734 [bsky, 1 points, 0 comments]
• The other is the complexity theory results cited as motivation in eg the HRM paper ( arxiv.org/abs/2506.21734 ) that show pretty clearly the range of problems that transformers simply can’t solve [bsky, 1 points, 1 comments]
• Vandaag helemaal in mijn sas met onverwachte leestip over meertraps doelredeneringen met AI. Hierarchical Reasoning Model -> arxiv.org/pdf/2506.21734 Gepubliceerd 22 juli, AI-ontwikkeling beweegt echt [bsky, 1 points, 2 comments]
• "Hierarchical Reasoning Model" New models for thinking show huge progress. They solve tough problems with fewer resources than ever before, but people are cautious about the hype. Article | Discussion [bsky, 1 points, 0 comments]
• arxiv.org/abs/2506.21734 なんやこれすごい 超少ないパラメータと1000個の事例で「問題の解き方そのものを学習してしまう」マジな話だったらとしたら、根本的なゲームチェンジャーになりうるやん 今までのLLMは図書館みたいになって、制御は全部HRMが行う、みたいな未来もあるのか [bsky, 1 points, 1 comments]
• Hierarchical Reasoning Model arxiv.org/abs/2506.21734 I can't judge the article yet but what really bothers me right at the top of the abstract... "Reasoning, the process of devising and executing com [bsky, 1 points, 1 comments]
• 英語で読む気も起きないのは内緒の話でござる arxiv.org/abs/2506.21734 github.com/sapientinc/HRM [bsky, 1 points, 1 comments]
• Well your first assumption may be challenged by other new work arxiv.org/abs/2506.21734 [bsky, 1 points, 0 comments]
• Have said for some time that as we model what neuroscience tells us, we'll see exponential leaps in this tech. Here is the latest leap, and it may well be a big one. (Pre-print) [bsky, 0 points, 1 comments]
• 논문은 실제로 사용되는 자원의 양은 배제한다는 점을 명확하게 하고 보더라도 드는 비용은 비슷하지 않으려나… 또 논문은 핵심만 발표하는 경향이 있으니까, 저걸 발표한 회사가 어떤 응용을 했을지도 궁금하구… [bsky, 0 points, 0 comments]
• Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 https://news.ycombinator.com/item?id=44699452 [bsky, 0 points, 0 comments]
• Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 arxiv.org [bsky, 0 points, 0 comments]
• Sapients paper on the concept of Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 [bsky, 0 points, 0 comments]
• Sapients paper on the concept of Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 [comments] [4 points] [bsky, 0 points, 0 comments]
• 📰 Hierarchical Reasoning Model 🔗 https://arxiv.org/abs/2506.21734 💬 Discuss on HN [bsky, 0 points, 0 comments]
• Hierarchical Reasoning Model arxiv.org/abs/2506.217... [bsky, 0 points, 0 comments]
• I am not an #AI expert, but I stumbled on this paper and it seems like a nice breakthrough. arxiv.org/pdf/2506.21734 Essentially, throwing away the current state of the art based on transformers and m [bsky, 0 points, 0 comments]
• (1/5) With all the ongoing hype around LLMs, an interesting research paper - arxiv.org/pdf/2506.21734 and its associated Github repo - github.com/sapientinc/HRM kind of went a bit under the radar. It [bsky, 0 points, 1 comments]
• A new architecture for #DeepLearning models: "Hierarchical Reasoning Model" arxiv.org/abs/2506.21734 It can do quite a bit with a smaller set of parameters and training data. Code and examples: github [bsky, 0 points, 0 comments]
• 2506.21734] Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 [#PostSapiens [bsky, 0 points, 0 comments]
• Sapients paper on the concept of Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 (https://news.ycombinator.com/item?id=44699452) [bsky, 0 points, 0 comments]
• ..capabilities. Read more: https://arxiv.org/abs/2506.21734 (10/10) [bsky, 0 points, 1 comments]
• Sapients paper on the concept of Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 [bsky, 0 points, 0 comments]
• ⚡ Hackernews Top story: Sapients paper on the concept of Hierarchical Reasoning Model [bsky, 0 points, 0 comments]
• Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 (http://news.ycombinator.com/item?id=44699452) [bsky, 0 points, 0 comments]
• Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 (http://news.ycombinator.com/item?id=44699452) [bsky, 0 points, 0 comments]
• Sapients paper on the concept of Hierarchical Reasoning Model https://arxiv.org/abs/2506.21734 (https://news.ycombinator.com/item?id=44699452) [bsky, 0 points, 0 comments]
• The paper "Hierarchical Reasoning Model" is so far only published on arxiv.org... arxiv.org/pdf/2506.217... [bsky, 0 points, 0 comments]
• [2506.21734] Hierarchical Reasoning Model arxiv.org/abs/2506.21734 news.ycombinator.com/item?id=4469... #arXiv #LLM #LLM #MLmodels #reasoning [bsky, 0 points, 0 comments]
• Hierarchical Reasoning Model arxiv.org/pdf/2506.21734 lnkd.in/p/eE8vPvvT [bsky, 0 points, 0 comments]
• Papers In The Park 8 in Trinity Bellwoods Park. We’re covering: arxiv.org/pdf/2506.21734 [bsky, 0 points, 0 comments]
• Interesting neural topology here. arxiv.org/abs/2506.21734 [bsky, 0 points, 0 comments]
• The original HRM paper by Wang et al can be found here: arxiv.org/abs/2506.21734 [bsky, 0 points, 0 comments]
• What is thinking anyway? Hierarchical Reasoning Modelhttps://arxiv.org/abs/2506.21734 [bsky, 0 points, 0 comments]
• I'd totally missed "Hierarchical Reasoning Model" (HRM) and follow up papers like TRM. arxiv.org/abs/2506.21734 arxiv.org/abs/2510.04871 These are extremely small language models that have some impres [bsky, 0 points, 1 comments]
• You may notice that sometimes ChatGPT takes too long to "Think" - reasoning is now becoming a critical lever to drive competitive differentiation in GenAI products, and a group of singapore researcher [bsky, 0 points, 1 comments]
• Besser als Claude 3.5 oder Gemini 2.5 arxiv.org/abs/2506.21734 [bsky, 0 points, 0 comments]
• It’s the Napster moment for model weights: centralization collapses when “good-enough” becomes local. ∞ arxiv.org/abs/2506.2... [bsky, 0 points, 0 comments]
• Hierarchical reasoning sounds like a step forward in LLM architecture. arxiv.org/pdf/2506.21734 sapient.inc/models [bsky, 0 points, 0 comments]
• Не съм била права, имат си публикация хората arxiv.org/abs/2506.21734 [bsky, 0 points, 0 comments]
• https://arxiv.org/abs/2506.21734 この論文では、人間の脳の階層的処理に着想を得た新しい再帰的アーキテクチャであるHRMを提案しています。 HRMは、2つレベルの相互依存的な再帰モジュールを介して、中間プロセスの明示的な監督なしに、単一のフォワードパスで連続的な推論タスクを実行。 2700万のパラメータのみで、HRMはわずか1000のトレーニングサンプルを使用して、複 [bsky, 0 points, 0 comments]
• Sapients paper on the concept of Hierarchical Reasoning Model [bsky, 0 points, 0 comments]
• Sapients paper on the concept of Hierarchical Reasoning Model #HackerNews https://arxiv.org/abs/2506.21734 [bsky, 0 points, 0 comments]

Related

• Truly Assessing Fluid Intelligence of Large Language Models through Dynamic Reasoning Evaluation
• HRM-Agent: Training a recurrent reasoning model in dynamic environments using reinforcement learning
• Modeling Hierarchical Thinking in Large Reasoning Models
• Less is More: Recursive Reasoning with Tiny Networks
• From System 1 to System 2: A Survey of Reasoning Large Language Models