Concentration of measure for non-linear random matrices with applications to neural networks and non-commutative polynomials

2025/07/10 by Adamczak, Radosław
#60E15 #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Primary: 60B20 #Probability (math.PR) #Secondary: 68T07

paper · doi:10.48550/arxiv.2507.07625

Abstract

We prove concentration inequalities for several models of non-linear random matrices. As corollaries we obtain estimates for linear spectral statistics of the conjugate kernel of neural networks and non-commutative polynomials in (possibly dependent) random matrices.

Citations

• Global law of conjugate kernel random matrices with heavy-tailed weights
• Ultra high order cumulants and quantitative CLT for polynomials in Random Matrices
• Large deviations of the empirical spectral measure of supercritical sparse Wigner matrices
• Spectral Evolution and Invariance in Linear-width Neural Networks
• A Concentration of Measure and Random Matrix Approach to Large\n Dimensional Robust Statistics
• Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
• Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
• Random non-Abelian G-circulant matrices. Spectrum of random convolution operators on large finite groups
• Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Concentration inequalities for non-Lipschitz functions with bounded derivatives of higher order
• Characterization of Talagrand's Like Transportation-Cost Inequalities on the Real Line
• On the Distribution of the Roots of Certain Symmetric Matrices

Related

• Concentration of measure and generalized product of random vectors with an application to Hanson-Wright-like inequalities
• Deriving Matrix Concentration Inequalities from Kernel Couplings
• An Introduction to Matrix Concentration Inequalities
• Matrix Concentration Inequalities and Free Probability II. Two-sided Bounds and Applications
• Concentration of polynomial random matrices via Efron-Stein inequalities