Symmetry Learning for Function Approximation in Reinforcement Learning

2017/06/09 by Anuj Mahajan, Mahajan, Anuj, Theja Tulabandhula +1
Computer Science · Neuroscience · #Artificial Intelligence (cs.AI) #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural dynamics and brain function #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1706.02999

openalex publication_date 2017/06/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper we explore methods to exploit symmetries for ensuring sample efficiency in reinforcement learning (RL), this problem deserves ever increasing attention with the recent advances in the use of deep networks for complex RL tasks which require large amount of training data. We introduce a novel method to detect symmetries using reward trails observed during episodic experience and prove its completeness. We also provide a framework to incorporate the discovered symmetries for functional approximation. Finally we show that the use of potential based reward shaping is especially effective for our symmetry exploitation mechanism. Experiments on various classical problems show that our method improves the learning performance significantly by utilizing symmetry information.

Citations

• Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning

Related

• Symmetry Detection in Trajectory Data for More Meaningful Reinforcement Learning Representations
• Exploiting Symmetry in Dynamics for Model-Based Reinforcement Learning with Asymmetric Rewards
• Reinforcement Learning Using known Invariances
• SiT: Symmetry-Invariant Transformers for Generalisation in Reinforcement Learning
• Learning broken symmetries with approximate invariance