Correspondence between neuroevolution and gradient descent

2020/08/31 by Stephen Whitelam, Viktor Selin, Sang-Won Park +1
Computer Science · Mathematics · Physics and Astronomy · #Algorithm #Applied mathematics #Artificial intelligence #Artificial neural network #Computer science #Domain Adaptation and Few-Shot Learning #Function (biology) #Gradient descent #Limit (mathematics) #Machine Learning and Algorithms #Mathematical analysis #Mathematics #Neural Networks and Applications #Neuroevolution #Stochastic gradient descent #cond-mat.stat-mech #cs.NE

paper · pdf · doi:10.1038/s41467-021-26568-2

arxiv created 2021/09/10 · openalex publication_date 2021/11/02 · arxiv updated 2022/01/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations, for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.

Citations

• Learning representations by back-propagating errors
• Playing Atari with Deep Reinforcement Learning
• On the difficulty of training Recurrent Neural Networks
• Taylor expansion of the accumulated rounding error
• Equation of State Calculations by Fast Computing Machines
• Deep Learning in Neural Networks: An Overview
• Approximation by superpositions of a sigmoidal function
• Reversible self-assembly of patchy particles into monodisperse icosahedral clusters
• Monte Carlo sampling methods using Markov chains and their applications
• Backpropagation Applied to Handwritten Zip Code Recognition
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
• On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent
• Guided evolutionary strategies: Augmenting random search with surrogate gradients
• Evolution Strategies Converges to Finite Differences
• Genetic Algorithms