Gradient flow in the kernel learning problem

2025/06/10 by Yang Li, Li, Yang, Feng Ruan +1 · 1 citation
Computer Science · #Differential Geometry (math.DG) #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Neural Networks and Applications #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2506.08550

openalex publication_date 2025/06/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This is a sequel to our paper `On the kernel learning problem'. We identify a canonical choice of Riemannian gradient flow, to find the stationary points in the kernel learning problem. In the presence of Gaussian noise variables, this flow enjoys the remarkable property of having a continuous family of Lyapunov functionals, and the interpretation is the automatic reduction of noise. PS. We include an extensive discussion in the postcript explaining the comparison with the 2-layer neural networks. Readers looking for additional motivations are encouraged to read the postscript immediately following the introduction.

Citations

• On the kernel learning problem
• A Theory of Feature Learning in Kernel Models
• Six Lectures on Linearized Neural Networks
• Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Cited by

• A Compositional Kernel Model for Feature Learning

Related

• Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel
• Dynamical Behaviors of the Gradient Flows for In-Context Learning
• Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
• Kernel Approximation of Fisher-Rao Gradient Flows
• On a continuous time model of gradient descent dynamics and instability in deep learning