2025/05/10 by Haokun Zhu, Zongtai Li, Zhu, Haokun +10 · 1 citation
Computer Science · Psychology · #Advanced Neural Network Applications #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Robotics (cs.RO) #Social Robot Interaction and HRI
paper · pdf · doi:10.48550/arxiv.2505.06729
openalex publication_date 2025/05/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Vision-Language Models (VLMs) have been increasingly integrated into object navigation tasks for their rich prior knowledge and strong reasoning abilities. However, applying VLMs to navigation poses two key challenges: effectively representing complex environment information and determining when and how to query VLMs. Insufficient environment understanding and over-reliance on VLMs (e.g. querying at every step) can lead to unnecessary backtracking and reduced navigation efficiency, especially in continuous environments. To address these challenges, we propose a novel framework that constructs a multi-layer representation of the environment during navigation. This representation consists of viewpoint, object nodes, and room nodes. Viewpoints and object nodes facilitate intra-room exploration and accurate target localization, while room nodes support efficient inter-room planning. Building on this representation, we propose a novel two-stage navigation policy, integrating high-level planning guided by VLM reasoning with low-level VLM-assisted exploration to efficiently locate a goal object. We evaluated our approach on three simulated benchmarks (HM3D, RoboTHOR, and MP3D), and achieved state-of-the-art performance on both the success rate (\mathord\uparrow 7.1%) and navigation efficiency (\mathord\uparrow 12.5%). We further validate our method on a real robot platform, demonstrating strong robustness across 15 object navigation tasks in 10 different indoor environments. Project page is available at https://zwandering.github.io/STRIVE.github.io/ .