SO(2)-Equivariant Reinforcement Learning

2022/03/08 by Dian Wang, Wang, Dian, Robin Walters +3 · 18 citations
Computer Science · Physics and Astronomy · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Model Reduction and Neural Networks #Reinforcement Learning in Robotics #Robotics (cs.RO) #cs.RO

paper · pdf · doi:10.48550/arxiv.2203.04439

Published at ICLR 2022

arxiv created 2022/03/08 · openalex publication_date 2022/03/08 · arxiv updated 2022/03/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Equivariant neural networks enforce symmetry within the structure of their convolutional layers, resulting in a substantial improvement in sample efficiency when learning an equivariant or invariant function. Such models are applicable to robotic manipulation learning which can often be formulated as a rotationally symmetric problem. This paper studies equivariant model architectures in the context of Q-learning and actor-critic reinforcement learning. We identify equivariant and invariant characteristics of the optimal Q-function and the optimal policy and propose equivariant DQN and SAC algorithms that leverage this structure. We present experiments that demonstrate that our equivariant versions of DQN and SAC can be significantly more sample efficient than competing algorithms on an important class of robotic manipulation problems.

Cited by

• Symmetry-Aware Steering of Equivariant Diffusion Policies: Benefits and Limits
• Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power
• Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetry
• EfficientFlow: Efficient Equivariant Flow Policy Learning for Embodied AI
• Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments
• Residual Rotation Correction using Tactile Equivariance
• Reinforcement Learning Using known Invariances
• SymmGrid: Super-Scaling On-Robot Learning with Parallelized Symmetries and Egocentric-Exocentric Visual Perception
• Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning
• Local-Canonicalization Equivariant Graph Neural Networks for Sample-Efficient and Generalizable Swarm Robot Control
• Multi-Group Equivariant Augmentation for Reinforcement Learning in Robot Manipulation
• Constraint-Preserving Data Generation for Visuomotor Policy Learning
• SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space
• Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulation
• EquAct: An SE(3)-Equivariant Multi-Task Transformer for Open-Loop Robotic Manipulation
• 3D Equivariant Visuomotor Policy Learning via Spherical Projection
• Equivariant Eikonal Neural Networks: Grid-Free, Scalable Travel-Time Prediction on Homogeneous Spaces
• A Practical Guide for Incorporating Symmetry in Diffusion Policy

Related

• Equivariant Q Learning in Spatial Action Spaces
• Equivariant Offline Reinforcement Learning
• Approximate Equivariance in Reinforcement Learning
• Equivariant Reinforcement Learning under Partial Observability
• Group Equivariant Deep Reinforcement Learning