A Recurrent Latent Variable Model for Sequential Data

2015/06/07 by Jun‐Young Chung, Chung, Junyoung, Kyle Kastner +9 · 60 citations
Computer Science · #Generative Adversarial Networks and Image Synthesis #Topic Modeling #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.1506.02216

Abstract

In this paper, we explore the inclusion of latent random variables into the dynamic hidden state of a recurrent neural network (RNN) by combining elements of the variational autoencoder. We argue that through the use of high-level latent random variables, the variational RNN (VRNN)1 can model the kind of variability observed in highly structured sequential data such as natural speech. We empirically evaluate the proposed model against related sequential models on four speech datasets and one handwriting dataset. Our results show the important roles that latent random variables can play in the RNN dynamic hidden state.

Citations

• Auto-Encoding Variational Bayes

Cited by

• Label-free Industrial Fault Detection via Adversarial Inverse Reinforcement Learning: A System for Run-to-Failure Prognostics
• Generative Recursive Reasoning
• Inference-time Stochastic Refinement of GRU-Normalizing Flow for Real-time Video Motion Transfer
• Leveraging unlabelled data for generalizable neural population decoding
• Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions
• Variational Transformers for Diverse Response Generation
• Variational Sequential Labelers for Semi-Supervised Learning
• baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotemporal Modeling
• Learning on predictions: Fusing training and autoregressive inference for long-term spatiotemporal forecasts
• A Survey on Bayesian Deep Learning
• Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows
• Analysis of ODE2VAE with Examples
• Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model
• Variational Marginal Particle Filters
• Continual Learning in Recurrent Neural Networks
• Latent-Variable Generative Models for Data-Efficient Text Classification
• Bayesian Attention Modules
• Mind the Gap when Conditioning Amortised Inference in Sequential\n Latent-Variable Models
• Contrastive Variational Reinforcement Learning for Complex Observations
• Towards Identifiability of Hierarchical Temporal Causal Representation Learning
• Vector Quantization in the Brain: Grid-like Codes in World Models
• CAMNet: Leveraging Cooperative Awareness Messages for Vehicle Trajectory Prediction
• UF-RNN: Real-Time Adaptive Motion Generation Using Uncertainty-Driven Foresight Prediction
• Memory-Augmented Generative AI for Real-time Wireless Prediction in Dynamic Industrial Environments
• Curiosity-Driven Development of Action and Language in Robots Through Self-Exploration
• Deep Learning for Physical Processes: Incorporating Prior Scientific\n Knowledge
• Neural Bayesian Filtering
• Dynamical system reconstruction from partial observations using stochastic dynamics
• EgoTraj-Bench: Towards Robust Trajectory Prediction Under Ego-view Noisy Observations
• S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation
• Deep Temporal Sigmoid Belief Networks for Sequence Modeling
• Recurrent Independent Mechanisms
• On Predictive Information in RNNs
• Transferable Time-Series Forecasting under Causal Conditional Shift
• Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future
• Latent Matters: Learning Deep State-Space Models
• BRAID: Input-Driven Nonlinear Dynamical Modeling of Neural-Behavioral Data
• Hierarchical Variational Imitation Learning of Control Programs
• Online Variational Filtering and Parameter Learning
• Deep Variational Luenberger-type Observer for Stochastic Video Prediction
• Sequential Adversarial Anomaly Detection for One-Class Event Data
• SoundSpaces: Audio-Visual Navigation in 3D Environments
• Sequential Neural Processes
• Semi-supervised Sequential Generative Models
• Graph Generation with Variational Recurrent Neural Network
• A Classifying Variational Autoencoder with Application to Polyphonic Music Generation
• Contrastively Disentangled Sequential Variational Autoencoder
• Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks
• Memory and attention in deep learning
• MemoryKT: An Integrative Memory-and-Forgetting Method for Knowledge Tracing
• FairDRL-ST: Disentangled Representation Learning for Fair Spatio-Temporal Mobility Prediction
• Probabilistic Video Generation using Holistic Attribute Control
• L-GTA: Latent Generative Modeling for Time Series Augmentation
• Learning representations for multivariate time series with missing data\n using Temporal Kernelized Autoencoders
• Schr "odingeRNN: Generative Modeling of Raw Audio as a Continuously\n Observed Quantum State
• Variational Recurrent Models for Solving Partially Observable Control Tasks
• Structured Object-Aware Physics Prediction for Video Modeling and Planning
• Behavior Priors for Efficient Reinforcement Learning
• Hierarchical Video Generation for Complex Data
• Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF

Related

• Regularized Sequential Latent Variable Models with Adversarial Neural Networks
• Sequential Neural Models with Stochastic Layers
• Z-Forcing: Training Stochastic Recurrent Networks
• Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
• Variable Computation in Recurrent Neural Networks