Planning from Pixels in Atari with Learned Symbolic Representations

2020/12/16 by Andrea Dittadi, Dittadi, Andrea, Frederik K. Drachmann +3
Computer Science · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2012.09126

openalex publication_date 2020/12/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Width-based planning methods have been shown to yield state-of-the-art performance in the Atari 2600 domain using pixel input. One successful approach, RolloutIW, represents states with the B-PROST boolean feature set. An augmented version of RolloutIW, π-IW, shows that learned features can be competitive with handcrafted ones for width-based search. In this paper, we leverage variational autoencoders (VAEs) to learn features directly from pixels in a principled manner, and without supervision. The inference model of the trained VAEs extracts boolean features from pixels, and RolloutIW plans with these features. The resulting combination outperforms the original RolloutIW and human professional play on Atari 2600 and drastically reduces the size of the feature set.

Related

• Deep Policies for Width-Based Planning in Pixel Domains
• Planning with Pixels in (Almost) Real Time
• Is Policy Learning Overrated?: Width-Based Planning and Active Learning for Atari
• Width-based Lookaheads with Learnt Base Policies and Heuristics Over the Atari-2600 Benchmark
• Planning from Pixels in Environments with Combinatorially Hard Search Spaces