2024/07/19 by Zaiqiao Meng, Hao Zhou, Meng, Zaiqiao +3 · 1 citation
Computer Science · Engineering · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Spatial Cognition and Navigation #Speech and dialogue systems
paper · pdf · doi:10.48550/arxiv.2407.14133
openalex publication_date 2024/07/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Visual Language Models (VLMs) are essential for various tasks, particularly visual reasoning tasks, due to their robust multi-modal information integration, visual reasoning capabilities, and contextual awareness. However, existing \VLMs' visual spatial reasoning capabilities are often inadequate, struggling even with basic tasks such as distinguishing left from right. To address this, we propose the \ours model, designed to enhance the visual spatial reasoning abilities of VLMS. ZeroVLM employs Zero-1-to-3, a 3D reconstruction model for obtaining different views of the input images and incorporates a prompting mechanism to further improve visual spatial reasoning. Experimental results on four visual spatial reasoning datasets show that our \ours achieves up to 19.48% accuracy improvement, which indicates the effectiveness of the 3D reconstruction and prompting mechanisms of our ZeroVLM.