Dyson Equation for Correlated Linearizations and Test Error of Random Features Regression

2023/12/14 by Hugo Latourelle-Vigeant, Latourelle-Vigeant, Hugo, Elliot Paquette +1 · 1 citation
Computer Science · Physics and Astronomy · #FOS: Mathematics #Model Reduction and Neural Networks #Neural Networks and Applications #Probability (math.PR) #Statistics Theory (math.ST) #Topological and Geometric Data Analysis

paper · pdf · doi:10.48550/arxiv.2312.09194

openalex publication_date 2023/12/14 · openalex created_date 2023/12/16 · openalex updated_date 2026/07/28

Abstract

This paper develops some theory of the Dyson equation for correlated linearizations and uses it to solve a problem on asymptotic deterministic equivalent for the test error in random features regression. The theory developed for the correlated Dyson equation includes existence-uniqueness, spectral support bounds, and stability properties. This theory is new for constructing deterministic equivalents for pseudo-resolvents of a class of linearizations with correlated entries. In the application, this theory is used to give a deterministic equivalent of the test error in random features ridge regression, in a proportional scaling regime, wherein we have conditioned on both training and test datasets.

Cited by

• Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling

Related

• Dimension-free deterministic equivalents and scaling laws for random feature regression
• Quantitative deterministic equivalent of sample covariance matrices with a general dependence structure
• Random Features Model with General Convex Regularization: A Fine Grained Analysis with Precise Asymptotic Learning Curves
• Precise analysis of ridge interpolators under heavy correlations -- a Random Duality Theory view
• Random Features Outperform Linear Models: Effect of Strong Input-Label Correlation in Spiked Covariance Data