Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaboration

2024/12/24 by Zhixuan Shen, Shen, Zhixuan, Haonan Luo +7 · 4 citations
Computer Science · Engineering · #FOS: Computer and information sciences #Robotics (cs.RO) #Robotics and Automated Systems #Semantic Web and Ontologies #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.2412.18292

openalex publication_date 2024/12/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Understanding how humans cooperatively utilize semantic knowledge to explore unfamiliar environments and decide on navigation directions is critical for house service multi-robot systems. Previous methods primarily focused on single-robot centralized planning strategies, which severely limited exploration efficiency. Recent research has considered decentralized planning strategies for multiple robots, assigning separate planning models to each robot, but these approaches often overlook communication costs. In this work, we propose Multimodal Chain-of-Thought Co-Navigation (MCoCoNav), a modular approach that utilizes multimodal Chain-of-Thought to plan collaborative semantic navigation for multiple robots. MCoCoNav combines visual perception with Vision Language Models (VLMs) to evaluate exploration value through probabilistic scoring, thus reducing time costs and achieving stable outputs. Additionally, a global semantic map is used as a communication bridge, minimizing communication overhead while integrating observational results. Guided by scores that reflect exploration trends, robots utilize this map to assess whether to explore new frontier points or revisit history nodes. Experiments on HM3D_v0.2 and MP3D demonstrate the effectiveness of our approach. Our code is available at https://github.com/FrankZxShen/MCoCoNav.git.

Cited by

• From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models
• ConPoSe: LLM-Guided Contact Point Selection for Scalable Cooperative Object Pushing
• LLM-MCoX: Large Language Model-based Multi-robot Coordinated Exploration and Search
• Conversational Orientation Reasoning: Egocentric-to-Allocentric Navigation with Multimodal Chain-of-Thought

Related

• Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Models
• SayCoNav: Utilizing Large Language Models for Adaptive Collaboration in Decentralized Multi-Robot Navigation
• A Survey on Improving Human Robot Collaboration through Vision-and-Language Navigation
• Efficient Collaborative Navigation through Perception Fusion for Multi-Robots in Unknown Environments
• Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge