Variational Dynamic Mixtures

2020/10/20 by Qiu Chen, Qiu, Chen, Stephan Mandt +3
Computer Science · #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.2010.10403

openalex publication_date 2020/10/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep probabilistic time series forecasting models have become an integral part of machine learning. While several powerful generative models have been proposed, we provide evidence that their associated inference models are oftentimes too limited and cause the generative model to predict mode-averaged dynamics. Modeaveraging is problematic since many real-world sequences are highly multi-modal, and their averaged dynamics are unphysical (e.g., predicted taxi trajectories might run through buildings on the street map). To better capture multi-modality, we develop variational dynamic mixtures (VDM): a new variational family to infer sequential latent variables. The VDM approximate posterior at each time step is a mixture density network, whose parameters come from propagating multiple samples through a recurrent architecture. This results in an expressive multi-modal posterior approximation. In an empirical study, we show that VDM outperforms competing approaches on highly multi-modal datasets from different domains.

Citations

• A Recurrent Latent Variable Model for Sequential Data
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Cubature Kalman Filters
• Long Short-Term Memory
• State Space LSTM Models with Particle MCMC Inference
• Tree-Structured Recurrent Switching Linear Dynamical Systems for\n Multi-Scale Modeling
• Latent Normalizing Flows for Discrete Sequences
• Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep\n Feature Spaces
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data

Related

• Dynamic Gaussian Mixture based Deep Generative Model For Robust Forecasting on Sparse Multivariate Time Series
• Mixture Density Conditional Generative Adversarial Network Models\n (MD-CGAN)
• Dual reparametrized Variational Generative Model for Time-Series Forecasting
• Variational inference of latent state sequences using Recurrent Networks
• Scalable Dynamic Mixture Model with Full Covariance for Probabilistic Traffic Forecasting