Extrapolation and learning equations

2016/10/10 by Georg Martius, Christoph H. Lampert, Martius, Georg +1 · 25 citations
Computer Science · Mathematics · #Algorithm #Artificial intelligence #Computer science #Deep learning #Differentiable function #Extrapolation #Function (biology) #Gaussian Processes and Bayesian Inference #Machine learning #Mathematics #Neural Networks and Applications #Process (computing) #Regression #Regularization (linguistics) #Set (abstract data type) #acm:62J02 #acm:65D15 #acm:68T05 #acm:68T30 #acm:68T40 #cs.AI #cs.LG #msc:62J02 #msc:65D15 #msc:68T05 #msc:68T30 #msc:68T40

paper · pdf · doi:10.48550/arxiv.1610.02995

published in arXiv (Cornell University) (Cornell University) · 13 pages, 8 figures, 4 tables

arxiv created 2016/10/10 · openalex publication_date 2016/10/10 · arxiv updated 2016/10/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08

Abstract

In classical machine learning, regression is treated as a black box process of identifying a suitable function from a hypothesis set without attempting to gain insight into the mechanism connecting inputs and outputs. In the natural sciences, however, finding an interpretable function for a phenomenon is the prime goal as it allows to understand and generalize results. This paper proposes a novel type of function learning network, called equation learner (EQL), that can learn analytical expressions and is able to extrapolate to unseen domains. It is implemented as an end-to-end differentiable feed-forward network and allows for efficient gradient based training. Due to sparsity regularization concise interpretable expressions can be obtained. Often the true underlying source expression is identified.

Cited by

• When Good Equations Get Bad Scores: Improving Symbolic Regression Through Better Parameter Optimization
• Symbolic regression for defect interactions in 2D materials
• Optimized Architectures for Kolmogorov-Arnold Networks
• Extrapolation of Periodic Functions Using Binary Encoding of Continuous Numerical Values
• Softly Symbolifying Kolmogorov-Arnold Networks
• Physically Interpretable Interatomic Potentials via Symbolic Regression and Reinforcement Learning
• When is a System Discoverable from Data? Discovery Requires Chaos
• Decomposable Neuro Symbolic Regression
• Shared Symbolic Backbones for Physically Consistent Multi-Output Symbolic Regression
• Toward Neural-Network-Guided Program Synthesis and Verification
• Iterated Agent for Symbolic Regression
• MDBench: Benchmarking Data-Driven Methods for Model Discovery
• Knowledge Integration for Physics-informed Symbolic Regression Using Pre-trained Large Language Models
• Data-driven Discovery of Digital Twins in Biomedical Research
• C¹-approximation with rational functions and rational neural networks
• SymMatika: Structure-Aware Symbolic Discovery
• SymbolicGPT: A Generative Transformer Model for Symbolic Regression
• DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience
• NeuronSeek: On Stability and Expressivity of Task-driven Neurons
• Robust extrapolation using physics-related activation functions in neural networks for nuclear masses
• Interpretable Robotic Friction Learning via Symbolic Regression
• Discovering Symbolic Differential Equations with Symmetry Invariants
• UniSymNet: A Unified Symbolic Network Guided by Transformer
• Think like a Scientist: Physics-guided LLM Agent for Equation Discovery
• Neuro-Evolutionary Approach to Physics-Aware Symbolic Regression
• CTSR: Cartesian tensor-based sparse regression for data-driven discovery of high-dimensional invariant governing equations

Related

• Learning Equations for Extrapolation and Control
• Learning from the Kernel and the Range Space
• Informed Equation Learning
• Learning Symbolic Expressions via Gumbel-Max Equation Learner Networks
• GINN-LP: A Growing Interpretable Neural Network for Discovering Multivariate Laurent Polynomial Equations