Physics of Learning: A Lagrangian perspective to different learning paradigms

2025/09/25 by Siyuan Guo, Bernhard Schölkopf, Guo, Siyuan +1 · 1 voice
Computer Science · Psychology · #FOS: Computer and information sciences #Innovative Teaching and Learning Methods #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.2509.21049

openalex publication_date 2025/09/25 · arxiv published 2025/09/25 · arxiv updated 2025/09/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study the problem of building an efficient learning system. Efficient learning processes information in the least time, i.e., building a system that reaches a desired error threshold with the least number of observations. Building upon least action principles from physics, we derive classic learning algorithms, Bellman's optimality equation in reinforcement learning, and the Adam optimizer in generative models from first principles, i.e., the Learning Lagrangian. We postulate that learning searches for stationary paths in the Lagrangian, and learning algorithms are derivable by seeking the stationary trajectories.

Citations

• ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
• Reinforcement Learning from Human Feedback
• Dimension-free deterministic equivalents and scaling laws for random feature regression
• 4+3 Phases of Compute-Optimal Neural Scaling Laws
• Beyond neural scaling laws: beating power law scaling via data pruning
• Training language models to follow instructions with human feedback
• Scaling Laws for Neural Language Models
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Adam: A Method for Stochastic Optimization
• Playing Atari with Deep Reinforcement Learning
• Reinforcement Learning: An Introduction

Discussions

• Physics of Learning: A Lagrangian perspective to different learning paradigms [hn, 3 points, 0 comments]

Related

• Learning Principle of Least Action with Reinforcement Learning
• Least Action Principles and Well-Posed Learning Problems
• There Will Be a Scientific Theory of Deep Learning
• Learning principle and mathematical realization of the learning mechanism in the brain
• On the Role of Time in Learning