Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images
2025/12/19 by Yang, Wenhao, Xia, Yu, Huang, Jinlong +7
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences
paper · doi:10.48550/arxiv.2512.17306
Abstract
Recent advances in large Vision-Language Models (VLMs) have exhibited strong reasoning capabilities on complex visual tasks by thinking with images in their Chain-of-Thought (CoT), which is achieved by actively invoking tools to analyze visual inputs rather than merely perceiving them. However, existing models often struggle to reflect on and correct themselves when attempting incorrect reasoning trajectories. To address this limitation, we propose DRIM, a model that enables deep but reliable multi-turn reasoning when thinking with images in its multimodal CoT. Our pipeline comprises three stages: data construction, cold-start SFT and RL. Based on a high-resolution image dataset, we construct high-difficulty and verifiable visual question-answer pairs, where solving each task requires multi-turn tool calls to reach the correct answer. In the SFT stage, we collect tool trajectories as cold-start data, guiding a multi-turn reasoning pattern. In the RL stage, we introduce redundancy-penalized policy optimization, which incentivizes the model to develop a self-reflective reasoning pattern. The basic idea is to impose judgment on reasoning trajectories and penalize those that produce incorrect answers without sufficient multi-scale exploration. Extensive experiments demonstrate that DRIM achieves superior performance on visual understanding benchmarks.
Citations
- DeepEyesV2: Toward Agentic Multimodal Model
- Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
- Ovis2.5 Technical Report
- Thyme: Think Beyond Images
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification
- MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration
- Active-O3: Empowering Multimodal Large Language Models with Active Perception via GRPO
- VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selection
- Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought
- Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning
- Seed1.5-VL Technical Report
- DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understanding
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- CMMCoT: Enhancing Complex Multi-Image Comprehension via Multi-Modal Chain-of-Thought and Memory Augmentation
- Qwen2.5-VL Technical Report
- VISA: Retrieval Augmented Generation with Visual Source Attribution
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
- ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
- Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
- Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
- LLaVA-OneVision: Easy Visual Task Transfer
- Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models
- Ovis: Structural Embedding Alignment for Multimodal Large Language Model
- STAR: A Benchmark for Situated Reasoning in Real-World Videos
- How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites
- BioXP-0.5B: Explainable Medical-AI via RL-GRPO
- InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
- V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
- LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
- Improved Baselines with Visual Instruction Tuning
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
- Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
- Visual Instruction Tuning
- Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text
- Micrograph segmentations for DDEVD
- Sigmoid Loss for Language Image Pre-Training
- MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action
- Multimodal Chain-of-Thought Reasoning in Language Models
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- Large Language Models are Zero-Shot Reasoners
- Flamingo: a Visual Language Model for Few-Shot Learning
- BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
- Learning Transferable Visual Models From Natural Language Supervision
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Visual7W: Grounded Question Answering in Images
- GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
- VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning
- REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization
- Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning
Related