vix.ing · top · new · best · stats · spec

Linear equivalence of nonlinear recurrent neural networks

2026/04/26 by David G. Clark · 1 voice · 1 citation
#cond-mat.dis-nn #q-bio.NC

paper · pdf

Abstract

Large nonlinear recurrent neural networks with random couplings generate rich, potentially chaotic activity and are of interest in neuroscience and other fields. A key object encoding the structure of activity is the N × N covariance matrix. Recent work proposed an ansatz in which, at large N and for typical quenched couplings, this covariance matrix matches that of a linear network with the same couplings, driven by independent noise. We derive this ansatz using a two-site cavity method that gives access to the joint statistics of activities at a pair of sites without disorder averaging. Specifically, we decompose each unit's activity into a linear response to its local field and a nonlinear residual; using the cavity method, we show that cross covariances of residuals at distinct sites are strongly suppressed, so that the residuals act as independent noise driving a linear network. In an alternative derivation, we construct a self-consistent equation for the covariance matrix in which non-Gaussian contributions supply cross terms that, in a linear network, would correspond to an external drive. Higher-order cross-site moments admit a Wick decomposition into pairwise covariances at leading order, reducing them to the linear-equivalent ansatz. We confirm the results in simulations and discuss their neuroscience implications.

Citations

Cited by

Discussions

Related