2026/06/29 by Mufan Li, Jaume de Dios Pont, Mihai Nica +1
#math.PR #stat.ML
We study the squared singular value spectrum of a non-square product of independent real Gaussian matrices, equivalently the feature covariance spectrum of a deep linear neural network at initialization. Starting from the fixed-m covariance diffusion previously obtained in the proportional depth-width limit, we record an equivalent matrix realization, describe its affine invariance, and derive the interacting diffusion satisfied by its eigenvalues. We then take a second limit, sending m→∞ on the accelerated spectral clock τ=mt, which corresponds in this sequential construction to the relation dm/n→τ. We establish convergence of the empirical spectral measure path to a deterministic mean-field limit and derive a closed Burgers equation for its T-transform. Together with the proportional depth-width limit, these results give a rigorous sequential route from the deep non-square Gaussian product to the free log-normal limit of its feature covariance spectrum; for more general initial laws, the transform yields a free multiplicative convolution form. We further analyze the support of the free log-normal law, give a fixed point iteration for numerical evaluation and a formal Marchenko--Pastur approximation at small time, and use the limiting spectrum to predict the risk in a toy random feature model.