TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics

2025/10/08 by Yi Han, Enshen Zhou, Han, Yi +15 · 8 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Robot Manipulation and Learning #Robotics (cs.RO) #Robotics and Sensor-Based Localization

paper · pdf · doi:10.48550/arxiv.2510.07181

openalex publication_date 2025/10/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Vision-Language Models (VLMs) have shown remarkable capabilities in spatial reasoning, yet they remain fundamentally limited to qualitative precision and lack the computational precision required for real-world robotics. Current approaches fail to leverage metric cues from depth sensors and camera calibration, instead reducing geometric problems to pattern recognition tasks that cannot deliver the centimeter-level accuracy essential for robotic manipulation. We present TIGeR (Tool-Integrated Geometric Reasoning), a novel framework that transforms VLMs from perceptual estimators to geometric computers by enabling them to generate and execute precise geometric computations through external tools. Rather than attempting to internalize complex geometric operations within neural networks, TIGeR empowers models to recognize geometric reasoning requirements, synthesize appropriate computational code, and invoke specialized libraries for exact calculations. To support this paradigm, we introduce TIGeR-300K, a comprehensive tool-invocation-oriented dataset covering point transformations, pose estimation, and spatial compatibility verification, complete with tool invocation sequences and intermediate computations. Through a two-stage training pipeline combining supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT) with our proposed hierarchical reward design, TIGeR achieves SOTA performance on geometric reasoning benchmarks while demonstrating centimeter-level precision in real-world robotic manipulation tasks.

Citations

• Do You Have Freestyle? Expressive Humanoid Locomotion via Audio Control
• RoboTracer: Mastering Spatial Trace with Reasoning in Vision-Language Models for Robotics
• RoboOS-NeXT: A Unified Memory-based Framework for Lifelong, Scalable, and Robust Multi-Robot Collaboration
• From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance
• MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles
• DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
• MotionTrans: Human VR Data Enable Motion-Level Learning for Robotic Manipulation Policies
• Reinforced Visual Perception with Tools
• VisualTrans: A Benchmark for Real-World Visual Transformation Reasoning
• MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details
• RoboBrain 2.0 Technical Report
• Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs
• VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learning
• RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics
• Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors
• Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence
• VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction
• ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models
• AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving
• SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning
• Qwen3 Technical Report
• ZeroSearch: Incentivize the Search Capability of LLMs without Searching
• RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration
• WebThinker: Empowering Large Reasoning Models with Deep Research Capability
• Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning
• ReTool: Reinforcement Learning for Strategic Tool Use in LLMs
• ToRL: Scaling Tool-Integrated RL
• RoboFactory: Exploring Embodied Agent Collaboration with Compositional Constraints
• MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs
• VGGT: Visual Geometry Grounded Transformer
• HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
• START: Self-taught Reasoner with Tools
• RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
• Qwen2.5-VL Technical Report
• NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistants
• OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning
• Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools
• Search-o1: Agentic Search-Enhanced Large Reasoning Models
• RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation
• RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
• RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation
• MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flow
• NVILA: Efficient Frontier Visual Language Models
• Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection
• Cubify Anything: Scaling Indoor 3D Object Detection
• RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
• LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning
• Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models
• SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models
• Space-LLaVA: a Vision-Language Model Adapted to Extraterrestrial Applications
• SAM 2: Segment Anything in Images and Videos
• DotaMath: Decomposition of Thought with Code Assistance and Self-correction for Mathematical Reasoning
• Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
• Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs
• SpatialBot: Precise Spatial Understanding with Vision Language Models
• Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models
• RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics
• EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models
• RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation
• SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
• BLINK: Multimodal Large Language Models Can See but Not Perceive
• AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation
• SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors
• MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning
• SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
• MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception
• Chain of Code: Reasoning with a Language Model-Augmented Code Emulator
• Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
• ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
• Recognize Anything: A Strong Image Tagging Model
• GPT-4 Technical Report
• Annotated Point Clouds and Images from a Spatial-Semantic Perception Pipeline: Robotized Deconstruction at the Reference Construction Site, Aachen
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
• AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains
• Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
• GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
• Gemini Robotics: Bringing AI into the Physical World
• Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models
• π³: Permutation-Equivariant Visual Geometry Learning
• InstructERC: Reforming Emotion Recognition in Conversation with Multi-task Retrieval-Augmented Large Language Models

Cited by

• SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL
• Geometrically-Constrained Agent for Spatial Reasoning
• Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs
• BOP-ASK: Object-Interaction Reasoning for Vision-Language Models
• RoboOS-NeXT: A Unified Memory-based Framework for Lifelong, Scalable, and Robust Multi-Robot Collaboration
• Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain
• From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance
• DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
• SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them

Related

• From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation
• A Universal Semantic-Geometric Representation for Robotic Manipulation
• RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics
• Perceiving, Reasoning, Adapting: A Dual-Layer Framework for VLM-Guided Precision Robotic Manipulation
• Task-oriented Robotic Manipulation with Vision Language Models