rQdia: Regularizing Q-Value Distributions With Image Augmentation

2025/06/26 by Jing Bi, Lerman, Sam, Bi, Jing
Computer Science · #Image and Signal Denoising Methods #Medical Image Segmentation Techniques #AI in cancer detection

paper · pdf · doi:10.48550/arxiv.2506.21367

Abstract

rQdia regularizes Q-value distributions with augmented images in pixel-based deep reinforcement learning. With a simple auxiliary loss, that equalizes these distributions via MSE, rQdia boosts DrQ and SAC on 9/12 and 10/12 tasks respectively in the MuJoCo Continuous Control Suite from pixels, and Data-Efficient Rainbow on 18/26 Atari Arcade environments. Gains are measured in both sample efficiency and longer-term training. Moreover, the addition of rQdia finally propels model-free continuous control from pixels over the state encoding baseline.

Citations

• Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark
• Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• A Simple Framework for Contrastive Learning of Visual Representations
• Self-Supervised Learning of Pretext-Invariant Representations
• Dream to Control: Learning Behaviors by Latent Imagination
• Momentum Contrast for Unsupervised Visual Representation Learning
• Momentum Contrast for Unsupervised Visual Representation Learning
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• Robust Visual Domain Randomization for Reinforcement Learning
• Improving Sample Efficiency in Model-Free Reinforcement Learning from\n Images
• Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Soft Actor-Critic Algorithms and Applications
• Quantifying Generalization in Reinforcement Learning
• Learning Latent Dynamics for Planning from Pixels
• Learning Actionable Representations from Visual Observations
• Robot Learning in Homes: Improving Generalization and Reducing Dataset\n Bias
• DeepMind Control Suite
• Distributional Reinforcement Learning with Quantile Regression
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Parameter Space Noise for Exploration
• Time-Contrastive Networks: Self-Supervised Learning from Video
• Domain Randomization for Transferring Deep Neural Networks from\n Simulation to the Real World
• Bridging the Gap Between Value and Policy Based Reinforcement Learning
• Dueling Network Architectures for Deep Reinforcement Learning
• Prioritized Experience Replay
• Deep Reinforcement Learning with Double Q-learning
• ImageNet classification with deep convolutional neural networks
• High-Performance Neural Networks for Visual Object Classification

Related

• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• Reinforcement Learning with Augmented Data
• Revisiting Data Augmentation in Deep Reinforcement Learning
• Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation
• A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning