Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning

2025/09/26 by Zejun Li, Li, Zejun, Yingxiu Zhao +15
Computer Science · #AI-based Problem Solving and Planning #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Constraint Satisfaction and Optimization #Data Visualization and Analytics #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2509.22746

openalex publication_date 2025/09/26 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

Current visual reasoning methods mainly focus on exploring specific reasoning modes. Although improvements can be achieved in particular domains, they struggle to develop general reasoning capabilities. Inspired by this, we propose a novel adaptive reasoning paradigm, Mixture-of-Visual-Thoughts (MoVT), which unifies different reasoning modes within a single model and guides it to select the appropriate mode based on context. To achieve this, we introduce AdaVaR, a two-stage Adaptive Visual Reasoning learning framework: different modes are unified and learned during the supervised cold-start stage, and the mode selection capability is induced via an RL process with a carefully designed AdaGRPO algorithm. Extensive experiments show that AdaVaR effectively guides the model to learn and differentiate multiple modes and perform context-adaptive mode selection, achieving consistent improvement across various scenarios, highlighting MoVT as an effective solution for building general visual reasoning models.

Citations

• Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning
• Grounded Reinforcement Learning for Visual Reasoning
• Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning
• One RL to See Them All: Visual Triple Unified Reinforcement Learning
• GeoSense: Evaluating Identification and Application of Geometric Principles in Multimodal Reasoning
• InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
• SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
• VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model
• Grounded Chain-of-Thought for Multimodal Large Language Models
• R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization
• LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL
• Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
• Visual-RFT: Visual Reinforcement Fine-Tuning
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
• LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
• Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
• Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
• What's in the Image? A Deep-Dive into the Vision of Vision Language Models
• ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
• LLaVA-CoT: Let Vision Language Models Reason Step-by-Step
• GPT-4o System Card
• Improve Vision Language Model Chain-of-thought Reasoning
• We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?
• VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models
• Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models
• How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
• Are We on the Right Way for Evaluating Large Vision-Language Models?
• MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
• OmniCount: Multi-label Object Counting with Semantic-Geometric Priors
• Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
• G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
• Compositional Chain-of-Thought Prompting for Large Multimodal Models
• Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
• MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• Graph of Thoughts: Solving Elaborate Problems with Large Language Models
• Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic
• Evaluating Object Hallucination in Large Vision-Language Models
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• Self-Evaluation Guided Beam Search for Reasoning
• Self-Refine: Iterative Refinement with Self-Feedback
• Reflexion: Language Agents with Verbal Reinforcement Learning
• Large Language Models are Zero-Shot Reasoners
• HyperTree Proof Search for Neural Theorem Proving
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning

Related

• Interleaved Latent Visual Reasoning with Selective Perceptual Modeling
• Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
• Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models
• Learning Visual Reasoning Without Strong Priors
• Mixture of Rationale: Multi-Modal Reasoning Mixture for Visual Question Answering