InSpire: Vision-Language-Action Models with Intrinsic Spatial Reasoning

2025/05/20 by Ji Zhang, Zhang, Ji, Shihan Wu +11 · 10 citations
Computer Science · Social Sciences · #FOS: Computer and information sciences #Geographic Information Systems Studies #Multimodal Machine Learning Applications #Robotics (cs.RO) #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.2505.13888

openalex publication_date 2025/05/20 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

Leveraging pretrained Vision-Language Models (VLMs) to map language instruction and visual observations to raw low-level actions, Vision-Language-Action models (VLAs) hold great promise for achieving general-purpose robotic systems. Despite their advancements, existing VLAs tend to spuriously correlate task-irrelevant visual features with actions, limiting their generalization capacity beyond the training data. To tackle this challenge, we propose Intrinsic Spatial Reasoning (InSpire), a simple yet effective approach that mitigates the adverse effects of spurious correlations by boosting the spatial reasoning ability of VLAs. Specifically, InSpire redirects the VLA's attention to task-relevant factors by prepending the question "In which direction is the [object] relative to the robot?" to the language instruction and aligning the answer "right/left/up/down/front/back/grasped" and predicted actions with ground-truth. Notably, InSpire can be used as a plugin to enhance existing autoregressive VLAs, requiring no extra training data or interaction with other large models. Extensive experimental results in both simulation and real-world environments demonstrate the effectiveness and flexibility of our approach.

Citations

• Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
• Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models
• HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation
• SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
• FAST: Efficient Action Tokenization for Vision-Language-Action Models
• Qwen2.5 Technical Report
• RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulation
• CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision
• π₀: A Vision-Language-Action Flow Model for General Robot Control
• Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance
• RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
• Robotic Control via Embodied Chain-of-Thought Reasoning
• Dreamitate: Real-World Visuomotor Policy Learning via Video Generation
• OpenVLA: An Open-Source Vision-Language-Action Model
• Situational Awareness Matters in 3D Vision Language Reasoning
• SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
• Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance
• Octo: An Open-Source Generalist Robot Policy
• Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs
• Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models
• RT-H: Action Hierarchies Using Language
• Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
• SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
• Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation
• Large Language Models are Visual Reasoning Coordinators
• Open X-Embodiment: Robotic Learning Datasets and RT-X Models
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
• DINOv2: Learning Robust Visual Features without Supervision
• Sigmoid Loss for Language Image Pre-Training
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Diffusion policy: Visuomotor policy learning via action diffusion
• LLaMA: Open and Efficient Foundation Language Models
• RT-1: Robotics Transformer for Real-World Control at Scale
• Grounding Language with Visual Affordances over Unstructured Data
• Code as Policies: Language Model Programs for Embodied Control
• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
• Skill Induction and Planning with Latent Language
• Scene Graph Reasoning for Visual Question Answering
• Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions
• DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
• A Closer Look at Memorization in Deep Networks

Cited by

• CoTinyVLA: Chain-of-Thought Distillation for a Sub-Billion-Parameter Vision-Language-Action Model
• An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
• MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalization
• TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Tracking
• Pure Vision Language Action (VLA) Models: A Comprehensive Survey
• Survey of Vision-Language-Action Models for Embodied Manipulation
• Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation
• Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling
• Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trends
• FunReason: Enhancing Large Language Models' Function Calling via Self-Refinement Multiscale Loss and Automated Data Refinement
• Policy Contrastive Decoding for Robotic Foundation Models

Related

• From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation
• Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model
• RetoVLA: Reusing Register Tokens for Spatial Reasoning in Vision-Language-Action Models
• Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models
• Evo-0: Vision-Language-Action Model with Implicit Spatial Understanding