A Review of Uncertainty for Deep Reinforcement Learning

2022/08/18 by Owen Lockwood, Lockwood, Owen, Mei Si +1 · 4 citations
Computer Science · #Data Stream Mining Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2208.09052

openalex publication_date 2022/08/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Uncertainty is ubiquitous in games, both in the agents playing games and often in the games themselves. Working with uncertainty is therefore an important component of successful deep reinforcement learning agents. While there has been substantial effort and progress in understanding and working with uncertainty for supervised learning, the body of literature for uncertainty aware deep reinforcement learning is less developed. While many of the same problems regarding uncertainty in neural networks for supervised learning remain for reinforcement learning, there are additional sources of uncertainty due to the nature of an interactable environment. In this work, we provide an overview motivating and presenting existing techniques in uncertainty aware deep reinforcement learning. These works show empirical benefits on a variety of reinforcement learning tasks. This work serves to help to centralize the disparate results and promote future research in this area.

Cited by

• Deep Gaussian Process Proximal Policy Optimization
• DRL-Based Robust Multi-Timescale Anti-Jamming Approaches under State Uncertainty
• Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations
• Post-Hoc Split-Point Self-Consistency Verification for Efficient, Unified Quantification of Aleatoric and Epistemic Uncertainty in Deep Learning
• Toward Optimal Statistical Inference in Noisy Linear Quadratic Reinforcement Learning over a Finite Horizon

Related

• Estimating Risk and Uncertainty in Deep Reinforcement Learning
• Incorporating Explicit Uncertainty Estimates into Deep Offline Reinforcement Learning
• Diverse Priors for Deep Reinforcement Learning
• Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learning
• Direct Uncertainty Estimation in Reinforcement Learning