2021/06/26 by Yue Zhao, Chenzhuang Du, Zhao, Yue +5
Biochemistry, Genetics and Molecular Biology · Computer Science · Neuroscience · #Advanced Fluorescence Microscopy Techniques #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neuroscience and Neural Engineering #Reinforcement Learning in Robotics
paper · pdf · doi:10.48550/arxiv.2106.13970
openalex publication_date 2021/06/26 · openalex created_date 2022/11/03 · openalex updated_date 2026/07/28
In vision-based reinforcement learning (RL) tasks, it is prevalent to assign auxiliary tasks with a surrogate self-supervised loss so as to obtain more semantic representations and improve sample efficiency. However, abundant information in self-supervised auxiliary tasks has been disregarded, since the representation learning part and the decision-making part are separated. To sufficiently utilize information in auxiliary tasks, we present a simple yet effective idea to employ self-supervised loss as an intrinsic reward, called Intrinsically Motivated Self-Supervised learning in Reinforcement learning (IM-SSR). We formally show that the self-supervised loss can be decomposed as exploration for novel states and robustness improvement from nuisance elimination. IM-SSR can be effortlessly plugged into any reinforcement learning with self-supervised auxiliary objectives with nearly no additional cost. Combined with IM-SSR, the previous underlying algorithms achieve salient improvements on both sample efficiency and generalization in various vision-based robotics tasks from the DeepMind Control Suite, especially when the reward signal is sparse.