Learning with springs and sticks

2025/08/26 by Luis Mantilla Calderón, Alán Aspuru-Guzik, Calderón, Luis Mantilla +1 · 1 voice
#cs.LG

paper · pdf · doi:10.48550/arxiv.2508.19015

Abstract

Learning is a physical process. Here, we aim to study a simple dynamical system composed of springs and sticks capable of arbitrarily approximating any continuous function. The main idea of our work is to use the sticks to mimic a piecewise-linear approximation of the given function, use the potential energy of springs to encode a desired mean squared error loss function, and converge to a minimum-energy configuration via dissipation. We apply the proposed simulation system to regression tasks and show that its performance is comparable to that of multi-layer perceptrons. In addition, we study the thermodynamic properties of the system and find a relation between the free energy change of the system and its ability to learn an underlying data distribution. We empirically find a thermodynamic learning barrier for the system caused by the fluctuations of the environment, whereby the system cannot learn if its change in free energy hits such a barrier. We believe this simple model can help us better understand learning systems from a physical point of view.

Citations

• Training of Physical Neural Networks
• Neural Network Diffusion
• Further improving quantum algorithms for nonlinear differential equations via higher-order methods and rescaling
• Thermodynamic Computing System for AI Applications
• Speed Limits for Deep Learning
• Fast quantum algorithm for differential equations
• GenPhys: From Physical Processes to Generative Models
• Exponential Quantum Speedup in Simulating Coupled Classical Oscillators
• Thermodynamic AI and the fluctuation frontier
• Denoising Diffusion Probabilistic Models
• Scaling Laws for Neural Language Models
• Scalable Gradients for Stochastic Differential Equations
• Quantifying the Carbon Emissions of Machine Learning
• An Artificial Spiking Quantum Neuron
• Energy and Policy Considerations for Deep Learning in NLP
• Deep Learning Scaling is Predictable, Empirically
• A Survey of Neuromorphic Computing and Neural Networks in Hardware
• Optimization Methods for Large-Scale Machine Learning
• Optimization Methods for Large-Scale Machine Learning
• Quantum Algorithm for Systems of Linear Equations with Exponentially Improved Dependence on Precision
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Stochastic thermodynamics, fluctuation theorems and molecular machines
• A Fast Learning Algorithm for Deep Belief Nets
• Adaptive Mixtures of Local Experts
• Neuromorphic electronic systems
• “Neural” computation of decisions in optimization problems
• The thermodynamics of computation—a review
• Statistical-Mechanical Theory of Irreversible Processes. I. General Theory and Simple Applications to Magnetic and Conduction Problems
• A Stochastic Approximation Method

Discussions

• Excited to share our most recent work: “Learning with springs and sticks” — now on arXiv! Paper link: arxiv.org/abs/2508.19015 Project page (with animations & code): bestquark.github.io/springs-and-.. [bsky, 14 points, 1 comments]

Related

• Quantifying many-body learning far from equilibrium with representation learning
• Mean-field theory of learning: from dynamics to statics
• Learning Physics from Data: a Thermodynamic Interpretation
• On instabilities in neural network-based physics simulators
• A Free-Energy Principle for Representation Learning