Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs

2025/10/20 by Zhining Liu, Liu, Zhining, Ziyi Chen +20 · 3 citations
Computer Science · Neuroscience · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Neurobiology of Language and Bilingualism #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2510.17771

openalex publication_date 2025/10/20 · openalex created_date 2025/10/22 · openalex updated_date 2026/07/28

Abstract

Vision-Language Models (VLMs) achieve strong results on multimodal tasks such as visual question answering, yet they can still fail even when the correct visual evidence is present. In this work, we systematically investigate whether these failures arise from not perceiving the evidence or from not leveraging it effectively. By examining layer-wise attention dynamics, we find that shallow layers focus primarily on text, while deeper layers sparsely but reliably attend to localized evidence regions. Surprisingly, VLMs often perceive the visual evidence when outputting incorrect answers, a phenomenon we term ``seeing but not believing'' that widely exists in major VLM families. Building on this, we introduce an inference-time intervention that highlights deep-layer evidence regions through selective attention-based masking. It requires no training and consistently improves accuracy across multiple families, including LLaVA, Qwen, Gemma, and InternVL. These results show that VLMs encode reliable evidence internally but under-utilize it, making such signals explicit can bridge the gap between perception and reasoning, advancing the diagnostic understanding and reliability of VLMs.

Citations

• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• Enhancing Visual Reliance in Text Generation: A Bayesian Perspective on Mitigating Hallucination in Large Vision-Language Models
• On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generation
• Gemma 3 Technical Report
• PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training
• Words or Vision: Do Vision-Language Models Have Blind Faith in Text?
• Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
• Qwen2.5-VL Technical Report
• Re-ranking the Context for Multimodal Retrieval Augmented Generation
• MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation
• VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information
• Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models
• ALR²: A Retrieve-then-Reason Framework for Long-context Question Answering
• Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs
• The Llama 3 Herd of Models
• Mindful-RAG: A Study of Points of Failure in Retrieval Augmented Generation
• Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
• Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention
• Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly
• VLind-Bench: Measuring Language Priors in Large Vision-Language Models
• Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning
• IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding
• Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?
• Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
• Improved Baselines with Visual Instruction Tuning
• GPT-4 Technical Report
• InfographicVQA
• DocVQA: A Dataset for VQA on Document Images
• Towards VQA Models That Can Read
• Attention Is All You Need
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Cited by

• Detect Before You Leap: Mirage Detection in Vision-Language Models
• Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correction
• Continual Low-Rank Adapters for LLM-based Generative Recommender Systems

Related

• Can VLMs Recall Factual Associations From Visual References?
• GLIMPSE: Holistic Cross-Modal Explainability for Large Vision-Language Models
• MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs
• VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs
• Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning