Stationary covariance spectra of discrete-time non-normal random recurrent dynamics

2026/06/30 by Jacob A. Zavatone-Veth · 1 voice
Biochemistry, Genetics and Molecular Biology · Physics and Astronomy · #q-bio.NC #cond-mat.dis-nn

paper · pdf

Abstract

Principal component analysis is widely used to characterize structure in the dynamics of recurrent neural networks. For stationary noise-driven dynamics, the distribution of variance among the principal components is determined by the spectrum of the stationary covariance matrix. While the spectral properties of this matrix are well-understood for linear networks with normal synaptic weight matrices, our understanding of the stationary covariance spectrum for random non-normal dynamics remains incomplete. In this note, we use a free-probability approach to formally derive a closed functional equation for the moment generating function of the limiting stationary covariance spectrum of discrete-time dynamics with random non-normal Gaussian weights. This characterization allows us to analyze the behavior of tail eigenvalues in the critical regime. In contrast, applying the same approach to the analogous continuous-time dynamics leads to an infinite hierarchy of Schwinger-Dyson equations, rather than a closed scalar equation. We conclude with some comments regarding the relevance of these results to comparisons of models of non-normal dynamics to neural data.

Citations

• Linear equivalence of nonlinear recurrent neural networks
• Covariance spectrum in nonlinear recurrent neural networks
• Dynamically Learning to Integrate in Recurrent Neural Networks
• Scaling and renormalization in high-dimensional regression
• Progress on the study of the Ginibre ensembles II: GinOE and GinSE
• An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices with Polynomial Scalings
• Searching for long time scales without fine tuning
• Controllability Gramian Spectra of Random Networks
• Nonnormal amplification in random balanced neuronal networks
• Eigenvector statistics in non-Hermitian random matrix ensembles
• Chaos in Random Neural Networks

Discussions

• Stationary covariance spectra of discrete-time non-normal random recurrent dynamics https://arxiv.org/abs/2606.31944 https://www.infiniteawareness.online/ [bsky, 0 points, 0 comments]

Related

• Covariance spectrum in nonlinear recurrent neural networks
• Kinetic energy in random recurrent neural networks
• Non-orthogonal eigenvectors, fluctuation-dissipation relations and entropy production
• Linear equivalence of nonlinear recurrent neural networks
• Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product