Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraints

2025/06/10 by Sunil Kumar, Kumar, Sunil, Bowen Zhao +5 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Constraint Satisfaction and Optimization #Data Visualization and Analytics #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2506.14821

openalex publication_date 2025/06/10 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

Despite tremendous recent advances in large model reasoning ability, vision-language models (VLMs) still struggle with detailed visual reasoning, especially when compute resources are limited. To address this challenge, we draw inspiration from methods like Deepseek-r1 for VLMs and train smaller-scale models with Group Relative Policy Optimization (GRPO) to use external tools such as zoom. The greatest benefit is obtained with a combination of GRPO learning, a simple reward structure, a simplified tool-calling interface, allocating additional tokens to the result of the tool call, and a training data mix that over-represents visually difficult examples. Compared to similarly-sized baseline models, our method achieves better performance on some visual question-answering (VQA) tasks, thanks to the detailed visual information gathered from the external tool.

Citations

• DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understanding
• ToolRL: Reward is All Tool Learning Needs
• VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning
• VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model
• Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
• R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
• Words or Vision: Do Vision-Language Models Have Blind Faith in Text?
• Qwen2.5-VL Technical Report
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
• GPT-4o System Card
• Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
• How Well Can Vision Language Models See Image Details?
• Vision language models are blind: Failing to translate detailed visual features into words
• Dragonfly: Multi-Resolution Zoom-In Encoding Enhances Vision-Language Models
• TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding
• Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models
• InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning
• Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
• V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
• Qwen Technical Report
• API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
• MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action
• Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models
• Visual Programming: Compositional visual reasoning without training
• Revisiting, Benchmarking and Exploring API Recommendation: How Far Are We?
• Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning
• Towards VQA Models That Can Read
• OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles
• LLaVA-UHD v2: an MLLM Integrating High-Resolution Semantic Pyramid via Hierarchical Window Transformer

Cited by

• Zooming into Comics: Region-Aware RL Improves Fine-Grained Comic Understanding in Vision-Language Models

Related

• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models
• CLGRPO: Reasoning Ability Enhancement for Small VLMs
• AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition
• Probing Visual Language Priors in VLMs
• Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs