2023/05/02 by Alaa Saade, Steven Kapturowski, Saade, Alaa +15 · 1 citation
Computer Science · #Anomaly Detection Techniques and Applications #Artificial Intelligence in Games #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Time Series Analysis and Forecasting
paper · pdf · doi:10.48550/arxiv.2305.01521
openalex publication_date 2023/05/02 · openalex created_date 2023/05/04 · openalex updated_date 2026/07/28
We introduce Robust Exploration via Clustering-based Online Density Estimation (RECODE), a non-parametric method for novelty-based exploration that estimates visitation counts for clusters of states based on their similarity in a chosen embedding space. By adapting classical clustering to the nonstationary setting of Deep RL, RECODE can efficiently track state visitation counts over thousands of episodes. We further propose a novel generalization of the inverse dynamics loss, which leverages masked transformer architectures for multi-step prediction; which in conjunction with RECODE achieves a new state-of-the-art in a suite of challenging 3D-exploration tasks in DM-Hard-8. RECODE also sets new state-of-the-art in hard exploration Atari games, and is the first agent to reach the end screen in "Pitfall!".