Towards VQA Models That Can Read

2019/04/18 by Singh, Amanpreet, Natarajan, Vivek, Shah, Meet +5 · 274 citations
#Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.1904.08920

Abstract

Studies have shown that a dominant class of questions asked by visually impaired users on images of their surroundings involves reading text in the image. But today's VQA models can not read! Our paper takes a first step towards addressing this problem. First, we introduce a new "TextVQA" dataset to facilitate progress on this important problem. Existing datasets either have a small proportion of questions about text (e.g., the VQA dataset) or are too small (e.g., the VizWiz dataset). TextVQA contains 45,336 questions on 28,408 images that require reasoning about text to answer. Second, we introduce a novel model architecture that reads text in the image, reasons about it in the context of the image and the question, and predicts an answer which might be a deduction based on the text and the image or composed of the strings found in the image. Consequently, we call our approach Look, Read, Reason & Answer (LoRRA). We show that LoRRA outperforms existing state-of-the-art VQA models on our TextVQA dataset. We find that the gap between human performance and machine performance is significantly larger on TextVQA than on VQA 2.0, suggesting that TextVQA is well-suited to benchmark progress along directions complementary to VQA 2.0.

Cited by

• DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding
• Same or Not? Enhancing Visual Perception in Vision-Language Models
• VL-RouterBench: A Benchmark for Vision-Language Model Routing
• Rethinking Fine-Tuning: Unlocking Hidden Capabilities in Vision-Language Models
• GOTS: Greedy Orthogonal Token Selection for High-Resolution Vision-Language Models
• Towards Signboard-Oriented Visual Question Answering: ViSignVQA Dataset, Method and Benchmark
• Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimization
• UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
• MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning
• Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs
• Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering
• Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model
• StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
• Detect Before You Leap: Mirage Detection in Vision-Language Models
• MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models
• Few Tokens Matter: Entropy Guided Attacks on Vision-Language Models
• Your Reasoning Benchmark May Not Test Reasoning: Revealing Perception Bottleneck in Abstract Reasoning Benchmarks
• FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models
• D2Pruner: Debiased Importance and Structural Diversity for MLLM Token Pruning
• Selective LoRA for Visual Tokens and Attention Heads
• CASA: Cross-Attention over Self-Attention for Efficient Vision-Language Fusion
• IPCV: Information-Preserving Compression for MLLM Visual Encoders
• OpenView: Empowering MLLMs with Out-of-view VQA
• HyDRA: Hierarchical and Dynamic Rank Adaptation for Mobile Vision Language Model
• Sketch-in-Latents: Eliciting Unified Reasoning in MLLMs
• Collaborative Edge-to-Server Inference for Vision-Language Models
• Seeing Beyond Words: Self-Supervised Visual Learning for Multimodal Large Language Models
• The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs
• T5Gemma 2: Seeing, Reading, and Understanding Longer
• JMMMU-Pro: Image-based Japanese Multi-discipline Multimodal Understanding Benchmark via Vibe Benchmark Construction
• Optimizing the Adversarial Perturbation with a Momentum-based Adaptive Matrix
• HyperVL: An Efficient and Dynamic Multimodal Large Language Model for Edge Devices
• SuperCLIP: CLIP with Simple Classification Supervision
• MMhops-R1: Multimodal Multi-hop Reasoning
• DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model
• ViInfographicVQA: A Benchmark for Single and Multi-image Visual Question Answering on Vietnamese Infographics
• Benchmarking the Generality of Vision-Language-Action Models
• Image Tiling for High-Resolution Reasoning: Balancing Local Detail with Global Context
• Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding
• Visual Funnel: Resolving Contextual Blindness in Multimodal Large Language Models
• Multilingual VLM Training: Adapting an English-Trained VLM to French
• Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs
• InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Models
• Towards Lossless Ultimate Vision Token Compression for VLMs
• HybridToken-VLM: Hybrid Token Compression for Vision-Language Models
• FRIEDA: Benchmarking Multi-Step Cartographic Reasoning in Vision-Language Models
• All You Need Are Random Visual Tokens? Demystifying Token Pruning in VLLMs
• START: Spatial and Textual Learning for Chart Understanding
• VisChainBench: A Benchmark for Multi-Turn, Multi-Image Visual Reasoning Beyond Language Priors
• Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models
• EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecture
• Towards Cross-View Point Correspondence in Vision-Language Models
• Thinking with Programming Vision: Towards a Unified View for Thinking with Images
• Jina-VLM: Small Multilingual Vision Language Model
• DIQ-H: Evaluating Hallucination Persistence in VLMs Under Temporal Visual Degradation
• Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Models
• ViDiC: Video Difference Captioning
• Mitigating Intra- and Inter-modal Forgetting in Continual Learning of Unified Multimodal Models
• dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model
• VACoT: Rethinking Visual Data Augmentation with VLMs
• VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm
• Script: Graph-Structured and Query-Conditioned Semantic Token Pruning for Multimodal Large Language Models
• See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Models
• Better, Stronger, Faster: Tackling the Trilemma in MLLM-based Segmentation with Simultaneous Textual Mask Prediction
• Visual Generation Tuning
• Quantized-Tinyllava: a new multimodal foundation model enables efficient split learning
• VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstruction
• From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoning
• Unexplored flaws in multiple-choice VQA evaluations
• DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Action
• EM-KD: Distilling Efficient Multimodal Large Language Model with Unbalanced Vision Tokens
• Text-Guided Semantic Image Encoder
• LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight
• Object-Centric Vision Token Pruning for Vision Language Models
• CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception
• Harmonious Parameter Adaptation in Continual Visual Instruction Tuning for Safety-Aligned MLLMs
• INTERLACE: Interleaved Layer Pruning and Efficient Adaptation in Large Vision-Language Models
• Robot-Powered Data Flywheels: Deploying Robots in the Wild for Continual Data Collection and Foundation Model Adaptation
• Parallel Vision Token Scheduling for Fast and Accurate Multimodal LMMs Inference
• Understanding Task Transfer in Vision-Language Models
• Towards Efficient VLMs: Information-Theoretic Driven Compression via Adaptive Structural Pruning
• AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
• Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models
• SMILE: A Composite Lexical-Semantic Metric for Question-Answering Evaluation
• VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning
• Learning to Think Fast and Slow for Visual Language Models
• Explore How to Inject Beneficial Noise in MLLMs
• Multimodal Evaluation of Russian-language Architectures
• Multimodal Continual Instruction Tuning with Dynamic Gradient Guidance
• A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models
• MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping
• Bias in, Bias out: Annotation Bias in Multilingual Large Language Models
• Draft and Refine with Visual Experts
• The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation
• Visual Spatial Tuning
• ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai
• NVIDIA Nemotron Nano V2 VL
• What's in Common? Multimodal Models Hallucinate When Reasoning Across Scenes
• Text-VQA Aug: Pipelined Harnessing of Large Multimodal Models for Automated Synthesis
• RegionRAG: Region-level Retrieval-Augmented Generation for Visual Document Understanding
• Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench
• SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understanding
• Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention
• VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs
• Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
• RL makes MLLMs see better than SFT
• Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation
• SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs
• BLM₁: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
• Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
• MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understanding
• Revisiting Multimodal Positional Encoding in Vision-Language Models
• Positional Preservation Embedding for Multimodal Large Language Models
• Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigation
• KBE-DME: Dynamic Multimodal Evaluation via Knowledge Enhanced Benchmark Evolution
• ARGenSeg: Image Segmentation with Autoregressive Image Generation Model
• GranViT: A Fine-Grained Vision Model With Autoregressive Perception For MLLMs
• HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Models
• Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Models
• TowerVision: Understanding and Improving Multilinguality in Vision-Language Models
• MedReason-R1: Learning to Reason for CT Diagnosis with Reinforcement Learning and Local Zoom
• CARES: Context-Aware Resolution Selector for VLMs
• Unified Reinforcement and Imitation Learning for Vision-Language Models
• Structured and Abstractive Reasoning on Multi-modal Relational Knowledge Images
• ProCLIP: Progressive Vision-Language Alignment via LLM-based Embedder
• DeepSeek-OCR: Contexts Optical Compression
• Online In-Context Distillation for Low-Resource Vision Language Models
• Accelerating Vision Transformers with Adaptive Patch Sizes
• SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inference
• Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs
• Token-Level Inference-Time Alignment for Vision-Language Models
• FineVision: Open Data Is All You Need
• VisiPruner: Decoding Discontinuous Cross-Modal Dynamics for Efficient Multimodal LLMs
• ZSPAPrune: Zero-Shot Prompt-Aware Token Pruning for Vision-Language Models
• Train a Unified Multimodal Data Quality Classifier with Synthetic Data
• From Pixels to Words -- Towards Native Vision-Language Primitives at Scale
• Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch?
• Spatial Preference Rewarding for MLLMs Spatial Understanding
• Vision-Centric Activation and Coordination for Multimodal Large Language Models
• InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue
• Seeing and Knowing in the Wild: Open-domain Visual Entity Recognition with Large-scale Knowledge Graphs via Contrastive Learning
• Document Intelligence in the Era of Large Language Models: A Survey
• End-to-End Multi-Modal Diffusion Mamba
• InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy
• CompoDistill: Attention Distillation for Compositional Reasoning in Multimodal LLMs
• ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution
• Scaling Language-Centric Omnimodal Representation Learning
• FlexAC: Towards Flexible Control of Associative Reasoning in Multimodal Large Language Models
• Data or Language Supervision: What Makes CLIP Better than DINO?
• FG-CLIP 2: A Bilingual Fine-grained Vision-Language Alignment Model
• Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey
• UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generation
• Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization
• Decoupled DiLoCo for Resilient Distributed Pre-training
• CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization
• Task-Aware Resolution Optimization for Visual Large Language Models
• Towards Efficient Multimodal Unified Reasoning Model via Model Merging
• Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping
• MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning
• How to Teach Large Multimodal Models New Skills
• To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
• NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints
• ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models
• Exploring OCR-augmented Generation for Bilingual VQA
• Conditional Representation Learning for Customized Tasks
• Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting
• Efficient Test-Time Scaling for Small Vision-Language Models
• Dirichlet-Prior Shaping: Guiding Expert Specialization in Upcycled MoEs
• Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Models
• Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs
• Efficient Multi-modal Large Language Models via Progressive Consistency Distillation
• Data Selection for Fine-tuning Vision Language Models via Cross Modal Alignment Trajectories
• Expert Merging: Model Merging with Unsupervised Expert Alignment and Importance-Guided Layer Chunking
• From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
• OIG-Bench: A Multi-Agent Annotated Benchmark for Multimodal One-Image Guides Understanding
• Training-Free Token Pruning via Zeroth-Order Gradient Estimation in Vision-Language Models
• Vision Function Layer in Multimodal LLMs
• When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs
• Skip-It? Theoretical Conditions for Layer Skipping in Vision-Language Models
• AutoPrune: Each Complexity Deserves a Pruning Policy
• HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
• PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications
• LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
• RCI: A Score for Evaluating Global and Local Reasoning in Multimodal Benchmarks
• Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional
• Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflection
• Uncovering Intrinsic Capabilities: A Paradigm for Data Curation in Vision-Language Models
• GaussianVision: Vision-Language Alignment from Compressed Image Representations using 2D Gaussian Splatting
• From Bias to Balance: Exploring and Mitigating Spatial Bias in LVLMs
• DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning
• LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference
• Unifying Adversarially Robust Model Experts in Vision-Language Models
• Energy-Driven Adaptive Visual Token Pruning for Efficient Vision-Language Models
• Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs
• VISA: Group-wise Visual Token Selection and Aggregation via Graph Summarization for Efficient MLLMs Inference
• PoRe: Position-Reweighted Visual Token Pruning for Vision Language Models
• From Global to Local: Social Bias Transfer in CLIP
• Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards
• OmniBridge: Unified Multimodal Understanding, Generation, and Retrieval via Latent Space Alignment
• Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models
• SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models
• RealBench: A Chinese Multi-image Understanding Benchmark Close to Real-world Scenarios
• Catching the Details: Self-Distilled RoI Predictors for Fine-Grained MLLM Perception
• When Big Models Train Small Ones: Label-Free Model Parity Alignment for Efficient Visual Question Answering using Small VLMs
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
• Pyramid Token Pruning for High-Resolution Large Vision-Language Models via Region, Token, and Instruction-Guided Importance
• Qianfan-VL: Domain-Enhanced Universal Vision-Language Models
• MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models
• AToken: A Unified Tokenizer for Vision
• ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decoding
• Mind the (Language) Gap: Towards Probing Numerical and Cross-Lingual Limits of LVLMs
• HERO: Rethinking Visual Token Early Dropping in High-Resolution Large Vision-Language Models
• AsyMoE: Leveraging Modal Asymmetry for Enhanced Expert Specialization in Large Vision-Language Models
• CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models
• 3D Aware Region Prompted Vision Language Model
• MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUs
• Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models
• PATIMT-Bench: A Multi-Scenario Benchmark for Position-Aware Text Image Machine Translation in Large Vision-Language Models
• Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations
• VARCO-VISION-2.0 Technical Report
• LaV-CoT: Language-Aware Visual CoT with Multi-Aspect Reward Optimization for Real-World Multilingual VQA
• Towards Understanding Visual Grounding in Visual Language Models
• Towards Reliable and Interpretable Document Question Answering via VLMs
• Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning
• OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
• VLMs-in-the-Wild: Bridging the Gap Between Academic Benchmarks and Enterprise Reality
• MoPEQ: Mixture of Mixed Precision Quantized Experts
• Draw-In-Mind: Rebalancing Designer-Painter Roles in Unified Multimodal Models Benefits Image Editing
• A Keyframe-Based Approach for Auditing Bias in YouTube Shorts Recommendations
• Improving Large Vision and Language Models by Learning from a Panel of Peers
• OpenVision 2: A Family of Generative Pretrained Visual Encoders for Multimodal Learning
• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models
• Fusion to Enhance: Fusion Visual Encoder to Enhance Multimodal Language Model
• TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Models
• Safe-LLaVA: A Privacy-Preserving Vision-Language Dataset and Benchmark for Biometric Safety
• CAD2DMD-SET: Synthetic Generation Tool of Digital Measurement Device CAD Model Datasets for fine-tuning Large Vision-Language Models
• Improving Alignment in LVLMs with Debiased Self-Judgment
• GRAFT: GRaPH and Table Reasoning for Textual Alignment -- A Benchmark for Structured Instruction Following and Visual Reasoning
• KRETA: A Benchmark for Korean Reading and Reasoning in Text-Rich VQA Attuned to Diverse Visual Contexts
• Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs
• Survey of Vision-Language-Action Models for Embodied Manipulation
• Directed-Tokens: A Robust Multi-Modality Alignment Approach to Large Language-Vision Models
• VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark
• Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation
• Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping
• Ovis2.5 Technical Report
• ORBIT: An Object Property Reasoning Benchmark for Visual Inference Tasks
• LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit
• MoIIE: Mixture of Intra- and Inter-Modality Experts for Large Vision Language Models
• DocThinker: Explainable Multimodal Large Language Models with Rule-based Reinforcement Learning for Document Understanding
• Reinforcement Learning for Large Model: A Survey
• Segmenting and Understanding: Region-aware Semantic Attention for Fine-grained Image Quality Assessment with Large Language Models
• MolmoAct: Action Reasoning Models that can Reason in Space
• AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning
• DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding
• BASIC: Boosting Visual Alignment with Intrinsic Refined Embeddings in Multimodal Large Language Models
• AdaptInfer: Adaptive Token Pruning for Vision-Language Model Inference with Dynamical Text Guidance
• Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models
• LoRA in LoRA: Towards Parameter-Efficient Architecture Expansion for Continual Visual Instruction Tuning
• LLaVA-RE: Binary Image-Text Relevancy Evaluation with Multimodal Large Language Model
• VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization
• S2M3: Split-and-Share Multi-Modal Models for Distributed Multi-Task Inference on the Edge
• Gather and Trace: Rethinking Video TextVQA from an Instance-oriented Perspective
• VLMQ: Efficient Post-Training Quantization for Large Vision-Language Models via Hessian Augmentation
• HiPrune: Training-Free Visual Token Pruning via Hierarchical Attention in Vision-Language Models
• Optimizing Vision-Language Consistency via Cross-Layer Regional Attention Alignment
• MoCHA: Advanced Vision-Language Reasoning with MoE Connector and Hierarchical Group Attention
• MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
• METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Models
• TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model
• Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression
• Region-based Cluster Discrimination for Visual Representation Learning

Related

• Making the V in Text-VQA Matter
• Grounding Answers for Visual Questions Asked by Visually Impaired People
• Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
• VQA: Visual Question Answering
• Visual question answering: from early developments to recent advances -- a survey