2021/03/08 by Tommaso Salvatori, Yuhang Song, Salvatori, Tommaso +8 · 1 voice · 5 citations
Computer Science · Engineering · Mathematics · Neuroscience · #Advanced Memory and Neural Computing #Algorithm #Artificial intelligence #Artificial neural network #Backpropagation #Coding (social sciences) #Computational neuroscience #Computer science #Deep learning #Inference #Machine learning #Mathematics #Neural Networks and Applications #Neural dynamics and brain function #Perceptron #Predictive coding #Recurrent neural network
paper · pdf · doi:10.48550/arxiv.2103.04689
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2021/03/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Deep learning has redefined the field of artificial intelligence (AI) thanks to the rise of artificial neural networks, which are architectures inspired by their neurological counterpart in the brain. Through the years, this dualism between AI and neuroscience has brought immense benefits to both fields, allowing neural networks to be used in dozens of applications. These networks use an efficient implementation of reverse differentiation, called backpropagation (BP). This algorithm, however, is often criticized for its biological implausibility (e.g., lack of local update rules for the parameters). Therefore, biologically plausible learning methods that rely on predictive coding (PC), a framework for describing information processing in the brain, are increasingly studied. Recent works prove that these methods can approximate BP up to a certain margin on multilayer perceptrons (MLPs), and asymptotically on any other complex model, and that zero-divergence inference learning (Z-IL), a variant of PC, is able to exactly implement BP on MLPs. However, the recent literature shows also that there is no biologically plausible method yet that can exactly replicate the weight update of BP on complex models. To fill this gap, in this paper, we generalize (PC and) Z-IL by directly defining them on computational graphs, and show that it can perform exact reverse differentiation. What results is the first biologically plausible algorithm that is equivalent to BP in the way of updating parameters on any neural network, providing a bridge between the interdisciplinary research of neuroscience and deep learning.