Parameter Estimation with Dense and Convolutional Neural Networks Applied to the FitzHugh-Nagumo ODE

2020/12/12 by Johann Rudi, Rudi, Johann, Julie Bessac +3 · 6 citations
Computer Science · Mathematics · Neuroscience · Physics and Astronomy · #Dynamical Systems (math.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks #Neural dynamics and brain function #cs.LG #math.DS #stat.ML #stochastic dynamics and bifurcation

paper · pdf · doi:10.48550/arxiv.2012.06691

accepted to MSML21: Mathematical and Scientific Machine Learning

openalex publication_date 2020/12/12 · arxiv created 2021/05/04 · arxiv updated 2021/05/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Machine learning algorithms have been successfully used to approximate nonlinear maps under weak assumptions on the structure and properties of the maps. We present deep neural networks using dense and convolutional layers to solve an inverse problem, where we seek to estimate parameters of a FitzHugh-Nagumo model, which consists of a nonlinear system of ordinary differential equations (ODEs). We employ the neural networks to approximate reconstruction maps for model parameter estimation from observational data, where the data comes from the solution of the ODE and takes the form of a time series representing dynamically spiking membrane potential of a biological neuron. We target this dynamical model because of the computational challenges it poses in an inference setting, namely, having a highly nonlinear and nonconvex data misfit term and permitting only weakly informative priors on parameters. These challenges cause traditional optimization to fail and alternative algorithms to exhibit large computational costs. We quantify the prediction errors of model parameters obtained from the neural networks and investigate the effects of network architectures with and without the presence of noise in observational data. We generalize our framework for neural network-based reconstruction maps to simultaneously estimate ODE parameters and parameters of autocorrelated observational noise. Our results demonstrate that deep neural networks have the potential to estimate parameters in dynamical models and stochastic processes, and they are capable of predicting parameters accurately for the FitzHugh-Nagumo model.

Citations

• A quantitative description of membrane current and its application to conduction and excitation in nerve
• Impulses and Physiological States in Theoretical Models of Nerve Membrane
• Automated neuron model optimization techniques: a review
• Deep Learning in Neural Networks: An Overview
• ImageNet classification with deep convolutional neural networks
• On the Selection of Initialization and Activation Function for Deep Neural Networks
• Sigmoid-weighted linear units for neural network function approximation in reinforcement learning
• Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs

Cited by

• Machine learning approach to single-shot multiparameter estimation for the non-linear Schrödinger equation
• Interpretable neural network system identification method for two families of second-order systems based on characteristic curves
• Comparative study of Bayesian and Frequentist methods for epidemic forecasting: Insights from simulated and historical data
• Theoretical guarantees for neural estimators in parametric statistics
• NOBLE -- Neural Operator with Biologically-informed Latent Embeddings to Capture Experimental Variability in Biological Neuron Models
• Fast Likelihood-Free Parameter Estimation for Lévy Processes
• PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networks

Related

• Neural Networks for Bayesian Inverse Problems Governed by a Nonlinear ODE
• Are Statistical Methods Obsolete in the Era of Deep Learning? A Study of ODE Inverse Problems
• Neural ODEs with Irregular and Noisy Data
• Physically constrained neural networks to solve the inverse problem for neuron models
• Learning via nonlinear conjugate gradients and depth-varying neural ODEs