The Neural Moving Average Model for Scalable Variational Inference of State Space Models

2019/10/02 by Tom Ryder, Dennis Prangle, Ryder, Tom +5 · 1 citation
Computer Science · Engineering · Mathematics · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Fault Detection and Control Systems #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.1910.00879

openalex publication_date 2019/10/02 · openalex created_date 2021/08/02 · openalex updated_date 2026/07/28

Abstract

Variational inference has had great success in scaling approximate Bayesian inference to big data by exploiting mini-batch training. To date, however, this strategy has been most applicable to models of independent data. We propose an extension to state space models of time series data based on a novel generative model for latent temporal states: the neural moving average model. This permits a subsequence to be sampled without drawing from the entire distribution, enabling training iterations to use mini-batches of the time series at low computational cost. We illustrate our method on autoregressive, Lotka-Volterra, FitzHugh-Nagumo and stochastic volatility models, achieving accurate parameter estimation in a short time.

Cited by

• Nested ensemble Kalman filter for static parameter inference in nonlinear state-space models

Related

• eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling
• Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series
• Note on Equivalence Between Recurrent Neural Network Time Series Models\n and Variational Bayesian Models
• Scalable Bayesian Learning for State Space Models using Variational Inference with SMC Samplers
• Variational inference of latent state sequences using Recurrent Networks