Intuition emerges in Maximum Caliber models at criticality

2025/08/08 by Arola-Fernández, Lluís
#Artificial Intelligence (cs.AI) #Disordered Systems and Neural Networks (cond-mat.dis-nn) #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (cs.LG) #Physics and Society (physics.soc-ph) #Statistical Mechanics (cond-mat.stat-mech)

paper · doi:10.48550/arxiv.2508.06477

Abstract

Whether large predictive models merely parrot their training data or produce genuine insight lacks a physical explanation. This work reports a primitive form of intuition that emerges as a metastable phase of learning that critically balances next-token prediction against future path-entropy. The intuition mechanism is discovered via mind-tuning, the minimal principle that imposes Maximum Caliber in predictive models with a control temperature-like parameter λ. Training on random walks in deterministic mazes reveals a rich phase diagram: imitation (low λ), rule-breaking hallucination (high λ), and a fragile in-between window exhibiting strong protocol-dependence (hysteresis) and multistability, where models spontaneously discover novel goal-directed strategies. These results are captured by an effective low-dimensional theory and frame intuition as an emergent property at the critical balance between memorizing what is and wondering what could be.

Citations

• The Missing Reward: Active Inference in the Era of Experience
• AlphaEvolve: A coding agent for scientific and algorithmic discovery
• Leveraging chaotic transients in the training of artificial neural networks
• Small Language Models are the Future of Agentic AI
• Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning
• Parameter Symmetry Potentially Unifies Deep Learning Theory
• Intrinsic motivation as constrained entropy maximization
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Roadmap on Neuromorphic Photonics
• Superhuman performance of a large language model on the reasoning tasks of a physician
• Intelligence at the Edge of Chaos
• Theoretical framework for quantum associative memories
• An effective theory of collective deep learning
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Designing ecosystems of intelligence from first principles
• The debate over understanding in AI’s large language models
• Complex behavior from intrinsic motivation to occupy action-state path space
• Training Compute-Optimal Large Language Models
• Maximum Entropy RL (Provably) Solves Some Robust RL Problems
• On the Dangers of Stochastic Parrots
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• The large learning rate phase of deep learning: the catapult mechanism
• Scaling Laws for Neural Language Models
• Machine learning and the physical sciences
• Reconciling modern machine learning practice and the bias-variance trade-off
• Attention Is All You Need
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Statistical physics of inference: Thresholds and algorithms
• Principles of maximum entropy and maximum caliber in statistical physics

Related

• The Illusion of Insight in Reasoning Models
• Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy Principle
• Representing Knowledge as Predictions (and State as Knowledge)
• What is Intelligence? A Cycle Closure Perspective
• Understanding Human Limits in Pattern Recognition: A Computational Model of Sequential Reasoning in Rock, Paper, Scissors