Variational Deep Q Network

2017/11/30 by Yunhao Tang, Alp Kucukelbir, Tang, Yunhao +1
Computer Science · #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Data Stream Mining Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1711.11225

openalex publication_date 2017/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a framework that directly tackles the probability distribution of the value function parameters in Deep Q Network (DQN), with powerful variational inference subroutines to approximate the posterior of the parameters. We will establish the equivalence between our proposed surrogate objective and variational inference loss. Our new algorithm achieves efficient exploration and performs well on large scale chain Markov Decision Process (MDP).

Citations

• Bootstrapped Thompson Sampling and Deep Exploration
• Dueling Network Architectures for Deep Reinforcement Learning
• Deep Exploration via Bootstrapped DQN
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Deep Probabilistic Programming
• Noisy Networks for Exploration
• Variational Inference: A Review for Statisticians

Related

• Exploring Variational Deep Q Networks
• Amortized Variational Deep Q Network
• Universal Approximation Theorem of Deep Q-Networks
• Distributionally Robust Deep Q-Learning
• Efficient Exploration through Bayesian Deep Q-Networks