Particle Filter Recurrent Neural Networks

2019/05/30 by Xiao Ma, Ma, Xiao, Péter Karkus +5 · 1 citation
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.1905.12885

openalex publication_date 2019/05/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data. To tackle highly variable and noisy real-world data, we introduce Particle Filter Recurrent Neural Networks (PF-RNNs), a new RNN family that explicitly models uncertainty in its internal structure: while an RNN relies on a long, deterministic latent state vector, a PF-RNN maintains a latent state distribution, approximated as a set of particles. For effective learning, we provide a fully differentiable particle filter algorithm that updates the PF-RNN latent state distribution according to the Bayes rule. Experiments demonstrate that the proposed PF-RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and 10 real-world sequence prediction datasets for text classification, stock price prediction, etc.

Citations

• Auto-Encoding Variational Bayes
• Long Short-Term Memory
• Bayesian Recurrent Neural Networks
• Deep Variational Reinforcement Learning for POMDPs

Cited by

• Model-Based Reinforcement Learning under Random Observation Delays

Related

• Hidden State Approximation in Recurrent Neural Networks Using Continuous Particle Filtering
• Predictive State Recurrent Neural Networks
• Differentiable and Stable Long-Range Tracking of Multiple Posterior Modes
• Recurrent Attentive Neural Process for Sequential Data
• Differentiable Particle Filters: End-to-End Learning with Algorithmic\n Priors