2023/03/08 by Julian Lemmel, Radu Grosu, Lemmel, Julian +1
Computer Science · Neuroscience · #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Neural Networks and Applications #Neural Networks and Reservoir Computing #Neural and Evolutionary Computing (cs.NE) #Neural dynamics and brain function #Systems and Control (eess.SY) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2303.04944
openalex publication_date 2023/03/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The approximation capability of ANNs and their RNN instantiations, is strongly correlated with the number of parameters packed into these networks. However, the complexity barrier for human understanding, is arguably related to the number of neurons and synapses in the networks, and to the associated nonlinear transformations. In this paper we show that the use of biophysical synapses, as found in LTCs, have two main benefits. First, they allow to pack more parameters for a given number of neurons and synapses. Second, they allow to formulate the nonlinear-network transformation, as a linear system with state-dependent coefficients. Both increase interpretability, as for a given task, they allow to learn a system linear in its input features, that is smaller in size compared to the state of the art. We substantiate the above claims on various time-series prediction tasks, but we believe that our results are applicable to any feedforward or recurrent ANN.