Increasing Computation Resolves Conflicts in Vision Language Models

2025/05/25 by Bingyang Wang, Luo, Dezhi, Yijiang Li +10 · 3 citations
Engineering · #Closed captioning #Cognition #Computation #Computational model #Construct (python library) #FOS: Computer and information sciences #Flexibility (engineering) #Mirroring #Models of neural computation #Neural and Evolutionary Computing (cs.NE) #Proxy (statistics) #Robotics and Automated Systems

paper · pdf · doi:10.48550/arxiv.2505.18969

published in ArXiv.org

openalex publication_date 2025/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Cognitive control, the ability to coordinate competing information sources in pursuit of goals, is fundamental to intelligent behavior. We systematically investigate whether Vision Language Models (VLMs) exhibit cognitive control and how computational resources modulate conflict resolution. We construct a benchmark of 4,410 tasks across seven conflict paradigms (Stroop, Flanker, and five realistic variants) spanning multiple difficulty levels and visual complexities, testing 47 VLMs with rigorous experimental control. We find that VLMs exhibit robust congruency effects across all tasks, with larger models systematically resolving conflicts more effectively than smaller models. Critically, VLMs reproduce the fine-grained demand-resource relationship observed in human temporal dynamics: larger models drop below chance on incongruent high-conflict trials while smaller models fail to meaningfully engage and perform at chance, mirroring human behavior at short processing times and establishing parameter count as a proxy for conflict resolution capacity. These findings demonstrate that human-like cognitive control emerges from optimization dynamics in large-scale neural networks, suggesting that adaptive flexibility under conflict may naturally arise through scaling.

Citations

• Robust Multimodal Large Language Models Against Modality Conflict
• Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models
• Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models
• Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
• Qwen2.5-VL Technical Report
• How the Stroop Effect Arises from Optimal Response Times in Laterally Connected Self-Organizing Maps
• LLaVA-CoT: Let Vision Language Models Reason Step-by-Step
• Core Knowledge Deficits in Multi-Modal Language Models
• The temporal dynamics of visual attention.
• MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?
• LLaVA-OneVision: Easy Visual Task Transfer
• SpatialBot: Precise Spatial Understanding with Vision Language Models
• VLind-Bench: Measuring Language Priors in Large Vision-Language Models
• Natural Language Can Help Bridge the Sim2Real Gap
• Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning
• SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
• V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
• LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge
• VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
• Evaluating Object Hallucination in Large Vision-Language Models
• A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues
• Visual Instruction Tuning
• GPT-4 Technical Report
• Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment
• TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
• The role of prefrontal cortex in cognitive control and executive function
• DocVQA: A Dataset for VQA on Document Images
• Scaling Laws for Neural Language Models
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• Reaction time in differential and developmental research: A review and commentary on the problems and alternatives.
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Adversarial examples in the physical world
• Executive Functions
• On the control of automatic processes: A parallel distributed processing account of the Stroop effect.
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning
• An Integrative Theory of Prefrontal Cortex Function
• Effects of noise letters upon the identification of a target letter in a nonsearch task

Cited by

• Video Models Start to Solve Chess, Maze, Sudoku, Mental Rotation, and Raven' Matrices
• Conflict Adaptation in Vision-Language Models
• A Very Big Video Reasoning Suite

Related

• Conflict Adaptation in Vision-Language Models
• Cognitive control as a multivariate optimization problem
• Neural dynamics of cognitive control: Current tensions and future promise
• Testing the Limits of Fine-Tuning for Improving Visual Cognition in Vision Language Models
• Visual cognition in multimodal large language models