2022/12/08 by Aidan Curtis, Curtis, Aidan, Leslie Pack Kaelbling +4 · 2 citations
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning and Data Classification #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO) #cs.RO
paper · pdf · doi:10.48550/arxiv.2212.04554
arxiv created 2022/12/08 · openalex publication_date 2022/12/08 · arxiv updated 2022/12/12 · openalex created_date 2022/12/25 · openalex updated_date 2026/07/28
Representing and reasoning about uncertainty is crucial for autonomous agents acting in partially observable environments with noisy sensors. Partially observable Markov decision processes (POMDPs) serve as a general framework for representing problems in which uncertainty is an important factor. Online sample-based POMDP methods have emerged as efficient approaches to solving large POMDPs and have been shown to extend to continuous domains. However, these solutions struggle to find long-horizon plans in problems with significant uncertainty. Exploration heuristics can help guide planning, but many real-world settings contain significant task-irrelevant uncertainty that might distract from the task objective. In this paper, we propose STRUG, an online POMDP solver capable of handling domains that require long-horizon planning with significant task-relevant and task-irrelevant uncertainty. We demonstrate our solution on several temporally extended versions of toy POMDP problems as well as robotic manipulation of articulated objects using a neural perception frontend to construct a distribution of possible models. Our results show that STRUG outperforms the current sample-based online POMDP solvers on several tasks.