2026/04/20 by Vin Bhaskara, Haicheng Wang · 1 voice
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Baseline (sea) #Convergence (economics) #Curiosity #Current (fluid) #Explainable Artificial Intelligence (XAI) #Focus (optics) #Mean squared prediction error #Noise (video) #Oracle #Stochastic Gradient Optimization Techniques #cs.AI #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2604.18701
openalex publication_date 2026/04/20 · arxiv published 2026/04/20 · openalex created_date 2026/04/23 · arxiv updated 2026/06/16 · openalex updated_date 2026/07/28
Local prediction-error-based curiosity rewards focus on the current transition without considering the world model's cumulative prediction error across all visited transitions. We introduce Curiosity-Critic, which grounds its intrinsic reward in the improvement of this cumulative objective, and show that it admits a tractable per-step surrogate: the difference between the current prediction error and the asymptotic error baseline of the current state transition. We estimate this error baseline online with a learned critic co-trained alongside the world model; since the critic only has to learn how hard a transition is to predict, its estimate of the irreducible noise floor converges well before the world model saturates, redirecting exploration toward learnable transitions. The reward is higher for learnable transitions and collapses toward zero for stochastic ones, thereby separating epistemic (reducible) from aleatoric (irreducible) prediction error online. Prior prediction-error curiosity formulations, from Schmidhuber (1991) to learned-feature-space variants, emerge as special cases corresponding to specific approximations of this error baseline. Experiments on a stochastic grid world show that Curiosity-Critic outperforms prediction-error, visitation-count, and Random Network Distillation methods in training speed and final world model accuracy.