2025/02/25 by Alice Guionnet, Guionnet, Alice, Vanessa Piccolo +1 · 3 citations
Mathematics · #15B52 #60B20 #68T07 #Advanced Algebra and Geometry #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Probability (math.PR) #Random Matrices and Applications #advanced mathematical theories
paper · pdf · doi:10.48550/arxiv.2502.18428
openalex created_date 2025/01/03 · openalex publication_date 2025/02/25 · openalex updated_date 2026/08/01
We study the asymptotic spectral distribution of the conjugate kernel random matrix YY^\top, where Y= f(WX) arises from a two-layer neural network model. We consider the setting where W and X are random rectangular matrices with i.i.d. entries, where the entries of W follow a heavy-tailed distribution, while those of X have light tails. Our assumptions on W include a broad class of heavy-tailed distributions, such as symmetric α-stable laws with α∈ ]0,2[ and sparse matrices with O(1) nonzero entries per row. The activation function f, applied entrywise, is bounded, smooth, odd, and nonlinear. We compute the limiting eigenvalue distribution of YY^\top through its moments and show that heavy-tailed weights induce strong correlations between the entries of Y, resulting in richer and fundamentally different spectral behavior compared to the light-tailed case.