DeepMind Control Suite

2018/01/02 by Yuval Tassa, Tassa, Yuval, Yotam Doron +21 · 2 voices · 135 citations
Computer Science · #Distributed and Parallel Computing Systems #Parallel Computing and Optimization Techniques #Reinforcement Learning in Robotics #cs.AI

paper · pdf · doi:10.48550/arxiv.1801.00690

Abstract

The DeepMind Control Suite is a set of continuous control tasks with a standardised structure and interpretable rewards, intended to serve as performance benchmarks for reinforcement learning agents. The tasks are written in Python and powered by the MuJoCo physics engine, making them easy to use and modify. We include benchmarks for several learning algorithms. The Control Suite is publicly available at https://www.github.com/deepmind/dm_control . A video summary of all tasks is available at http://youtu.be/rAai4QzcYbs .

Cited by

• Biased Dreams: Limitations to Epistemic Uncertainty Quantification in Latent Dynamics Models
• Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations
• ChronoDreamer: Action-Conditioned World Model as an Online Simulator for Robotic Planning
• Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applications
• Spectral Representation-based Reinforcement Learning
• Prismatic World Model: Learning Compositional Dynamics for Planning in Hybrid Systems
• Delay-Aware Diffusion Policy: Bridging the Observation-Execution Gap in Dynamic Tasks
• BiTAgent: A Task-Aware Modular Framework for Bidirectional Coupling between Multimodal Large Language Models and World Models
• AltNet: Addressing the Plasticity-Stability Dilemma in Reinforcement Learning
• Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning
• Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments
• Hardware-Software Collaborative Computing of Photonic Spiking Reinforcement Learning for Robotic Continuous Control
• SOMBRL: Scalable and Optimistic Model-Based RL
• ArticFlow: Generative Simulation of Articulated Mechanisms
• Dream to Control: Learning Behaviors by Latent Imagination
• COMBO: Conservative Offline Model-Based Policy Optimization
• Information Theoretic Model Predictive Q-Learning
• Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
• Measuring Progress in Deep Reinforcement Learning Sample Efficiency
• Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction
• Human-like Planning for Reaching in Cluttered Environments
• SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
• Auto-Tuned Sim-to-Real Transfer
• Online and Offline Reinforcement Learning by Planning with a Learned Model
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
• Data-Efficient Reinforcement Learning with Self-Predictive\n Representations
• Representation Learning Enables Scalable Multitask Deep Reinforcement Learning
• On Geometric Structures for Policy Parameterization in Continuous Control
• Dynamic Sparsity: Challenging Common Sparsity Assumptions for Learning World Models in Robotic Reinforcement Learning Benchmarks
• Learning to Focus: Prioritizing Informative Histories with Structured Attention Mechanisms in Partially Observable Reinforcement Learning
• HMSPC: A Hybrid Mechanistic-Stochastic Physical-Continuous Model for Battery Dynamics
• MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarios
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep\n Reinforcement Learning
• Dexterous Robotic Piano Playing at Scale
• Clustering-Based Weight Orthogonalization for Stabilizing Deep Reinforcement Learning
• Bootstrap Off-policy with World Model
• Off-policy Reinforcement Learning with Model-based Exploration Augmentation
• Planning to Explore via Self-Supervised World Models
• Arena: a toolkit for Multi-Agent Reinforcement Learning
• Interactive Differentiable Simulation
• Robust Predictable Control
• Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning
• Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
• Rigorous Agent Evaluation: An Adversarial Approach to Uncover\n Catastrophic Failures
• ARCHER: Aggressive Rewards to Counter bias in Hindsight Experience Replay
• Scalable Coordinated Exploration in Concurrent Reinforcement Learning
• To Distill or Decide? Understanding the Algorithmic Trade-off in Partially Observable Reinforcement Learning
• Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning
• PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inference
• Use the Online Network If You Can: Towards Fast and Stable Reinforcement Learning
• Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows
• Neural Co-state Policies: Structuring Hidden States in Recurrent Reinforcement Learning
• Hamilton-Jacobi Deep Q-Learning for Deterministic Continuous-Time Systems with Lipschitz Continuous Controls
• Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
• Invariant Causal Prediction for Block MDPs
• D2 Actor Critic: Diffusion Actor Meets Distributional Critic
• Sample-efficient and Scalable Exploration in Continuous-Time RL
• Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecification
• TARC: Time-Adaptive Robotic Control
• World Models Should Prioritize the Unification of Physical and Social Dynamics
• Q-Learning in enormous action spaces via amortized approximate maximization
• Exploring Conditions for Diffusion models in Robotic Control
• Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via\n Latent Model Ensembles
• Contrastive Variational Reinforcement Learning for Complex Observations
• Learning and Planning in Complex Action Spaces
• Towards Robust Zero-Shot Reinforcement Learning
• A Comprehensive Survey on World Models for Embodied AI
• Benchmarking Model-Based Reinforcement Learning
• Relative Entropy Regularized Policy Iteration
• Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learning
• Unsupervised Learning of Object Keypoints for Perception and Control
• FOSSIL: Harnessing Feedback on Suboptimal Samples for Data-Efficient Generalisation with Imitation Learning for Embodied Vision-and-Language Tasks
• SCOPED: Score-Curvature Out-of-distribution Proximity Evaluator for Diffusion
• Decoupling Representation Learning from Reinforcement Learning
• Population-Coded Spiking Neural Networks for High-Dimensional Robotic Control
• PAC-Bayesian Reinforcement Learning Trains Generalizable Policies
• Gaze on the Prize: Shaping Visual Attention with Return-Guided Contrastive Learning
• Co-design is powerful and not free
• Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraints
• Local Reinforcement Learning with Action-Conditioned Root Mean Squared Q-Functions
• BuilderBench -- A benchmark for generalist agents
• On the role of planning in model-based deep reinforcement learning
• Challenges of Real-World Reinforcement Learning
• Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators
• Geometric Properties of Neural Multivariate Regression
• TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning
• Combining Large Language Models and Gradient-Free Optimization for Automatic Control Policy Synthesis
• Noise-Guided Transport for Imitation Learning
• XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning
• Quantifying Generalisation in Imitation Learning
• DyMoDreamer: World Modeling with Dynamic Modulation
• Continuous-Discrete Reinforcement Learning for Hybrid Control in\n Robotics
• STAIR: Addressing Stage Misalignment through Temporal-Aligned Preference Reinforcement Learning
• Functional Critic Modeling for Provably Convergent Off-Policy Actor-Critic
• From Parameters to Behavior: Unsupervised Compression of the Policy Space
• Graph networks as learnable physics engines for inference and control
• Deterministic Implementations for Reproducibility in Deep Reinforcement\n Learning
• Generalization in Reinforcement Learning by Soft Data Augmentation
• TriFinger: An Open-Source Robot for Learning Dexterity
• Harnessing Structures for Value-Based Planning and Reinforcement Learning
• QQWorld: Quantile-Quantile Matching for World Model Regularization
• Latent Matters: Learning Deep State-Space Models
• Variance Reduction for Evolution Strategies via Structured Control Variates
• Pure Vision Language Action (VLA) Models: A Comprehensive Survey
• Sample-based Distributional Policy Gradient
• TF-Replicator: Distributed Machine Learning for Researchers
• Learning from Observation: A Survey of Recent Advances
• Robust Reinforcement Learning for Continuous Control with Model Misspecification
• Unity: A General Platform for Intelligent Agents
• Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation
• Momentum-based Accelerated Q-learning
• Critic Regularized Regression
• Neural Graph Evolution: Towards Efficient Automatic Robot Design
• Actor-Critic Reinforcement Learning for Control with Stability Guarantee
• Reinforcement Learning in the Wild with Maximum Likelihood-based Model Transfer
• Learning to Manipulate Deformable Objects without Demonstrations
• SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
• Accelerating Reinforcement Learning through GPU Atari Emulation
• Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization
• Online Hyper-parameter Tuning in Off-policy Learning via Evolutionary Strategies
• Search-Based Credit Assignment for Offline Preference-Based Reinforcement Learning
• Compute-Optimal Scaling for Value-Based Deep RL
• Categorical Policies: Multimodal Policy Learning and Exploration in Continuous Control
• DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneck
• Task-Agnostic Morphology Evolution
• Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances
• MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments
• Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challenges
• Robust Remote Reinforcement Learning over Unreliable Communication Channels using Homomorphic State Encoding
• Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies
• One-Step Flow Policy Mirror Descent
• CompILE: Compositional Imitation Learning and Execution
• Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks
• Model-Predictive Control via Cross-Entropy and Gradient-Based Optimization

Discussions

• DeepMind Control Suite [hn, 3 points, 0 comments]
• DeepMind Control Suite [hn, 1 points, 0 comments]

Related

• Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking
• Deep Reinforcement Learning
• DeepMind Lab2D
• URLB: Unsupervised Reinforcement Learning Benchmark
• DeepMind Lab