Probing the relationship between linear dynamical systems and low-rank\n recurrent neural network models

2021/10/19 by Adrian Valente, Srdjan Ostojic, Valente, Adrian +4 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · Neuroscience · Physics and Astronomy · #Algorithm #Artificial intelligence #Artificial neural network #Combinatorics #Computer science #Curse of dimensionality #Dynamical systems theory #FOS: Biological sciences #Gaussian Processes and Bayesian Inference #Linear dynamical system #Linear model #Linear system #Machine learning #Mathematics #Model Reduction and Neural Networks #Neural Networks and Applications #Neural dynamics and brain function #Neurons and Cognition (q-bio.NC) #Physics #Population #Projection (relational algebra) #Rank (graph theory) #Recurrent neural network #q-bio.NC

paper · pdf · doi:10.48550/arxiv.2110.09804

21 pages, 2 figures

arxiv created 2021/10/19 · openalex publication_date 2021/10/19 · arxiv updated 2021/10/20 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

A large body of work has suggested that neural populations exhibit\nlow-dimensional dynamics during behavior. However, there are a variety of\ndifferent approaches for modeling low-dimensional neural population activity.\nOne approach involves latent linear dynamical system (LDS) models, in which\npopulation activity is described by a projection of low-dimensional latent\nvariables with linear dynamics. A second approach involves low-rank recurrent\nneural networks (RNNs), in which population activity arises directly from a\nlow-dimensional projection of past activity. Although these two modeling\napproaches have strong similarities, they arise in different contexts and tend\nto have different domains of application. Here we examine the precise\nrelationship between latent LDS models and linear low-rank RNNs. When can one\nmodel class be converted to the other, and vice versa? We show that latent LDS\nmodels can only be converted to RNNs in specific limit cases, due to the\nnon-Markovian property of latent LDS models. Conversely, we show that linear\nRNNs can be mapped onto LDS models, with latent dimensionality at most twice\nthe rank of the RNN.\n

Citations

• Robust timing and motor patterns by taming chaos in recurrent neural networks
• Chaos in Random Neural Networks
• A New Approach to Linear Filtering and Prediction Problems

Cited by

• Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example

Related

• Inferring stochastic low-rank recurrent neural networks from neural data
• Linking connectivity, dynamics and computations in low-rank recurrent neural networks
• Linear dynamical neural population models through nonlinear embeddings
• A Disentangled Low-Rank RNN Framework for Uncovering Neural Connectivity and Dynamics
• Neural Dynamics Discovery via Gaussian Process Recurrent Neural Networks