Loss landscapes and optimization in over-parameterized non-linear systems and neural networks

2022/01/10 by Chaoyue Liu, Libin Zhu, Mikhail Belkin · 12 citations
Computer Science · Engineering · Physics and Astronomy · #Model Reduction and Neural Networks #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · doi:10.1016/j.acha.2021.12.009

openalex publication_date 2022/01/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Cited by

• Large-Time Analysis of the Langevin Dynamics for Energies Fulfilling Polyak-Łojasiewicz Conditions
• Discrete State Diffusion Models: A Sample Complexity Perspective
• Gradient Flow Convergence Guarantee for General Neural Network Architectures
• Explaining Grokking and Information Bottleneck through Neural Collapse Emergence
• Nonconvex Decentralized Stochastic Bilevel Optimization under Heavy-Tailed Noise
• Predictability Enables Parallelization of Nonlinear State Space Models
• Predator-Prey Model: Driven Hunt for Accelerated Grokking
• Convergence for adaptive resampling of random Fourier features
• Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation
• Extended Interface Physics-Informed Neural Networks Method for Moving Interface Problems
• From Sublinear to Linear: Fast Convergence in Deep Networks via Locally Polyak-Lojasiewicz Regions
• Concentration of measure for non-linear random matrices with applications to neural networks and non-commutative polynomials