Gradient flow on extensive-rank positive semi-definite matrix denoising

2023/03/16 by Antoine Bodin, Bodin, Antoine, Nicolas Macris +1
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Matrix Theory and Algorithms #Sparse and Compressive Sensing Techniques #Topological and Geometric Data Analysis

paper · pdf · doi:10.48550/arxiv.2303.09474

openalex publication_date 2023/03/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we present a new approach to analyze the gradient flow for a positive semi-definite matrix denoising problem in an extensive-rank and high-dimensional regime. We use recent linear pencil techniques of random matrix theory to derive fixed point equations which track the complete time evolution of the matrix-mean-square-error of the problem. The predictions of the resulting fixed point equations are validated by numerical experiments. In this short note we briefly illustrate a few predictions of our formalism by way of examples, and in particular we uncover continuous phase transitions in the extensive-rank and high-dimensional regime, which connect to the classical phase transitions of the low-rank problem in the appropriate limit. The formalism has much wider applicability than shown in this communication.

Related

• Rank-one matrix estimation: analytic time evolution of gradient descent dynamics
• Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem
• From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent
• Understanding Incremental Learning with Closed-form Solution to Gradient Flow on Overparamerterized Matrix Factorization
• Scaled stochastic gradient descent for low-rank matrix completion