ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

2022/03/19 by Ahmed Masry, Masry, Ahmed, Do Xuan Long +7 · 297 citations
Computer Science · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #Data Management and Algorithms #Data Visualization and Analytics #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2203.10244

openalex publication_date 2022/03/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Charts are very popular for analyzing data. When exploring charts, people often ask a variety of complex reasoning questions that involve several logical and arithmetic operations. They also commonly refer to visual features of a chart in their questions. However, most existing datasets do not focus on such complex reasoning questions as their questions are template-based and answers come from a fixed-vocabulary. In this work, we present a large-scale benchmark covering 9.6K human-written questions as well as 23.1K questions generated from human-written chart summaries. To address the unique challenges in our benchmark involving visual and logical reasoning over charts, we present two transformer-based models that combine visual features and the data table of the chart in a unified way to answer questions. While our models achieve the state-of-the-art results on the previous datasets as well as on our benchmark, the evaluation also reveals several challenges in answering complex reasoning questions.

Cited by

• DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding
• Same or Not? Enhancing Visual Perception in Vision-Language Models
• VL-RouterBench: A Benchmark for Vision-Language Model Routing
• Modality Inflation: Energy Characterization and Optimization Opportunities for MLLM Inference
• Self-Rewarded Multimodal Coherent Reasoning Across Diverse Visual Domains
• Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone
• Masking Teacher and Reinforcing Student for Distilling Vision-Language Models
• Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimization
• Offline-Online Curriculum RL for Multimodal Reasoning
• UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
• Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation
• PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
• Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model
• See Less, See Right: Bi-directional Perceptual Shaping For Multimodal Reasoning
• StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
• NEXT: Reasoning-Driven Video Recommendation via a Vision-Language Model
• Chart Deception in Vision-Language Models: From Vulnerability to Mitigation
• MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models
• CHaystack: Benchmarking Chinese Document Retrieval and VQA
• Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts
• PERELMAN: Pipeline for scientific literature meta-analysis. Technical report
• Cube Bench: A Benchmark for Spatial Visual Reasoning in MLLMs
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoning
• D2Pruner: Debiased Importance and Structural Diversity for MLLM Token Pruning
• CycleChart: A Unified Consistency-Based Learning Framework for Bidirectional Chart Understanding and Generation
• CASA: Cross-Attention over Self-Attention for Efficient Vision-Language Fusion
• OpenView: Empowering MLLMs with Out-of-view VQA
• FPBench: A Comprehensive Benchmark of Multimodal Large Language Models for Fingerprint Analysis
• Xiaomi MiMo-VL-Miloco Technical Report
• CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoning
• Differences That Matter: Auditing Models for Capability Gap Discovery and Rectification
• Seeing Beyond Words: Self-Supervised Visual Learning for Multimodal Large Language Models
• DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models
• T5Gemma 2: Seeing, Reading, and Understanding Longer
• SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understanding
• HyperVL: An Efficient and Dynamic Multimodal Large Language Model for Edge Devices
• ChartAgent: A Chart Understanding Framework with Tool Integrated Reasoning
• Reassessing the Role of Supervised Fine-Tuning: An Empirical Study in VLM Reasoning
• ViInfographicVQA: A Benchmark for Single and Multi-image Visual Question Answering on Vietnamese Infographics
• Image Tiling for High-Resolution Reasoning: Balancing Local Detail with Global Context
• Towards Fine-Grained Recognition with Large Visual Language Models: Benchmark and Optimization Strategies
• Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs
• InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Models
• Towards Lossless Ultimate Vision Token Compression for VLMs
• Thinking with Images via Self-Calling Agent
• FRIEDA: Benchmarking Multi-Step Cartographic Reasoning in Vision-Language Models
• START: Spatial and Textual Learning for Chart Understanding
• Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning
• VisChainBench: A Benchmark for Multi-Turn, Multi-Image Visual Reasoning Beyond Language Priors
• VG-Refiner: Towards Tool-Refined Referring Grounded Reasoning via Agentic Reinforcement Learning
• Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models
• Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs
• EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecture
• Jina-VLM: Small Multilingual Vision Language Model
• AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition
• Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Models
• PPTBench: Towards Holistic Evaluation of Large Language Models for PowerPoint Layout and Design Understanding
• VACoT: Rethinking Visual Data Augmentation with VLMs
• TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models
• Script: Graph-Structured and Query-Conditioned Semantic Token Pruning for Multimodal Large Language Models
• See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Models
• ChartAnchor: Chart Grounding with Structural-Semantic Fidelity
• ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning
• Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding
• From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images
• Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalization
• Structured Extraction from Business Process Diagrams Using Vision-Language Models
• CaptionQA: Is Your Caption as Useful as the Image Itself?
• Qwen3-VL Technical Report
• EM-KD: Distilling Efficient Multimodal Large Language Model with Unbalanced Vision Tokens
• LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight
• Boosting Reasoning in Large Multimodal Models via Activation Replay
• VeriSciQA: An Auto-Verified Dataset for Scientific Visual Question Answering
• HKRAG: Holistic Knowledge Retrieval-Augmented Generation Over Visually-Rich Documents
• Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs
• INTERLACE: Interleaved Layer Pruning and Efficient Adaptation in Large Vision-Language Models
• Understanding Task Transfer in Vision-Language Models
• VADE: Variance-Aware Dynamic Sampling via Online Sample-Level Difficulty Estimation for Multimodal RL
• DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translation
• AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
• ChainV: Atomic Visual Hints Make Multimodal Reasoning Shorter and Better
• EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards
• Learning to Think Fast and Slow for Visual Language Models
• Multimodal Evaluation of Russian-language Architectures
• ChartEditor: A Reinforcement Learning Framework for Robust Chart Editing
• A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models
• SciEGQA: A Dataset for Scientific Evidence-Grounded Question Answering and Reasoning
• MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping
• Direct Visual Grounding by Directing Attention of Visual Tokens
• Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data
• Simple Vision-Language Math Reasoning via Rendered Text
• The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation
• TabRAG: Tabular Document Retrieval via Structured Language Representations
• DeepEyesV2: Toward Agentic Multimodal Model
• SDS KoPub VDR: A Benchmark Dataset for Visual Document Retrieval in Korean Public Documents
• Visual Spatial Tuning
• Cambrian-S: Towards Spatial Supersensing in Video
• ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai
• NVIDIA Nemotron Nano V2 VL
• What's in Common? Multimodal Models Hallucinate When Reasoning Across Scenes
• SVG Decomposition for Enhancing Large Multimodal Models Visualization Comprehension: A Study with Floor Plans
• ChartM³: A Multi-Stage Code-Driven Pipeline for Constructing Multi-Dimensional and Multi-Step Visual Reasoning Data in Chart Comprehension
• Routing-Based Continual Learning for Multimodal Large Language Models
• UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings
• LongCat-Flash-Omni Technical Report
• Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning
• From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustration
• Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmark
• Masked Diffusion Captioning for Visual Feature Learning
• ChartAB: A Benchmark for Chart Grounding & Dense Alignment
• Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench
• SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understanding
• CATCH: A Modular Cross-domain Adaptive Template with Hook
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning
• ALDEN: Reinforcement Learning for Active Navigation and Evidence Gathering in Long Documents
• AnnoBench: A Benchmark for Visualization Annotation Generation
• ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding
• Decoupled Visual Processing: Efficient Multimodal Adaptation via Modality-Specific Transformer Substitution
• Understanding Knowledge Transfer Mechanism in Heterogeneous MLLM Fusion: A Simple Linear Approach
• ICDAR 2026 Competition on Information Extraction from Atomic Layer Deposition/Etching (ALD/E) Scientific Figures
• VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs
• RL makes MLLMs see better than SFT
• PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistencies
• Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation
• SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs
• JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence
• Revisiting Multimodal Positional Encoding in Vision-Language Models
• The Benchmarking Epistemology: Construct Validity for Evaluating Machine Learning Models
• A Survey of AI Scientists
• Positional Preservation Embedding for Multimodal Large Language Models
• OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM
• Beyond Reasoning Gains: Mitigating General Capabilities Forgetting in Large Reasoning Models
• KBE-DME: Dynamic Multimodal Evaluation via Knowledge Enhanced Benchmark Evolution
• ARGenSeg: Image Segmentation with Autoregressive Image Generation Model
• GranViT: A Fine-Grained Vision Model With Autoregressive Perception For MLLMs
• Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Models
• MedReason-R1: Learning to Reason for CT Diagnosis with Reinforcement Learning and Local Zoom
• CARES: Context-Aware Resolution Selector for VLMs
• Unified Reinforcement and Imitation Learning for Vision-Language Models
• Structured and Abstractive Reasoning on Multi-modal Relational Knowledge Images
• DeepSeek-OCR: Contexts Optical Compression
• SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inference
• FineVision: Open Data Is All You Need
• ZSPAPrune: Zero-Shot Prompt-Aware Token Pruning for Vision-Language Models
• MultiVerse: A Multi-Turn Conversation Benchmark for Evaluating Large Vision and Language Models
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding
• Composition-Grounded Instruction Synthesis for Visual Reasoning
• From Pixels to Words -- Towards Native Vision-Language Primitives at Scale
• Vision-Centric Activation and Coordination for Multimodal Large Language Models
• RECODE: Reasoning Through Code Generation for Visual Question Answering
• Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
• Diagnosing Bottlenecks in Data Visualization Understanding by Vision-Language Models
• CompoDistill: Attention Distillation for Compositional Reasoning in Multimodal LLMs
• MetaCaptioner: Towards Generalist Visual Captioning with Open-source Suites
• ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution
• HoneyBee: Data Recipes for Vision-Language Reasoners
• Scaling Language-Centric Omnimodal Representation Learning
• FlexAC: Towards Flexible Control of Associative Reasoning in Multimodal Large Language Models
• Chart-RVR: Reinforcement Learning with Verifiable Rewards for Explainable Chart Reasoning
• VOGUE: Guiding Exploration with Visual Uncertainty Improves Multimodal Reasoning
• ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models
• Decoupled DiLoCo for Resilient Distributed Pre-training
• Towards Efficient Multimodal Unified Reasoning Model via Model Merging
• MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning
• How to Teach Large Multimodal Models New Skills
• Optimizing delivery for quick commerce factoring qualitative assessment of generated routes
• ARM2: Adaptive Reasoning Model with Vision Understanding and Executable Code
• NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints
• Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices
• LAD-RAG: Layout-aware Dynamic RAG for Visually-Rich Document Understanding
• Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods
• ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models
• Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks
• Beyond Monolithic Rewards: A Hybrid and Multi-Aspect Reward Optimization for MLLM Alignment
• Factuality Matters: When Image Generation and Editing Meet Structured Visuals
• Exploring OCR-augmented Generation for Bilingual VQA
• RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning
• Efficient Test-Time Scaling for Small Vision-Language Models
• Uncovering the Computational Ingredients of Human-Like Representations in LLMs
• MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training
• MR²-Bench: Going Beyond Matching to Reasoning in Multimodal Retrieval
• Expert Merging: Model Merging with Unsupervised Expert Alignment and Importance-Guided Layer Chunking
• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play
• From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
• PixelCraft: A Multi-Agent System for High-Fidelity Visual Reasoning on Structured Images
• OIG-Bench: A Multi-Agent Annotated Benchmark for Multimodal One-Image Guides Understanding
• Vision Function Layer in Multimodal LLMs
• AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomy
• MRAG-Suite: A Diagnostic Evaluation Platform for Visual Retrieval-Augmented Generation
• HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
• PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications
• LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
• RCI: A Score for Evaluating Global and Local Reasoning in Multimodal Benchmarks
• RIV: Recursive Introspection Mask Diffusion Vision Language Model
• Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional
• Chimera: Diagnosing Shortcut Learning in Visual-Language Understanding
• Multilingual Vision-Language Models, A Survey
• SciTS: Scientific Time Series Understanding and Generation with LLMs
• ERGO: Efficient High-Resolution Visual Understanding for Vision-Language Models
• MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources
• DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning
• A Simple Data Augmentation Strategy for Text-in-Image Scientific VQA
• LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference
• QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding
• The Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation From Recognition to Reasoning
• Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards
• OmniBridge: Unified Multimodal Understanding, Generation, and Retrieval via Latent Space Alignment
• Visual Instruction Pretraining for Domain-Specific Foundation Models
• MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction
• Losing the Plot: How VLM responses degrade on imperfect charts
• Qwen3-Omni Technical Report
• ChartHal: A Fine-grained Framework Evaluating Hallucination of Large Vision Language Models in Chart Understanding
• Catching the Details: Self-Distilled RoI Predictors for Fine-Grained MLLM Perception
• When Big Models Train Small Ones: Label-Free Model Parity Alignment for Efficient Visual Question Answering using Small VLMs
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
• Pyramid Token Pruning for High-Resolution Large Vision-Language Models via Region, Token, and Instruction-Guided Importance
• Qianfan-VL: Domain-Enhanced Universal Vision-Language Models
• DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards
• Towards Rationale-Answer Alignment of LVLMs via Self-Rationale Calibration
• Explain Before You Answer: A Survey on Compositional Visual Reasoning
• HERO: Rethinking Visual Token Early Dropping in High-Resolution Large Vision-Language Models
• AsyMoE: Leveraging Modal Asymmetry for Enhanced Expert Specialization in Large Vision-Language Models
• MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe
• Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
• ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinement
• 3D Aware Region Prompted Vision Language Model
• MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUs
• Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models
• PATIMT-Bench: A Multi-Scenario Benchmark for Position-Aware Text Image Machine Translation in Large Vision-Language Models
• VARCO-VISION-2.0 Technical Report
• Towards Reliable and Interpretable Document Question Answering via VLMs
• Visual Programmability: A Guide for Code-as-Thought in Chart Understanding
• RADAR: A Reasoning-Guided Attribution Framework for Explainable Visual Data Analysis
• TextlessRAG: End-to-End Visual Document RAG by Speech Without Text
• Visual-TableQA: Open-Domain Benchmark for Reasoning over Table Images
• MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval
• OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
• VLMs-in-the-Wild: Bridging the Gap Between Academic Benchmarks and Enterprise Reality
• CMRAG: Co-modality-based visual document retrieval and question answering
• Draw-In-Mind: Rebalancing Designer-Painter Roles in Unified Multimodal Models Benefits Image Editing
• A-SEA3L-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation
• Improving Large Vision and Language Models by Learning from a Panel of Peers
• Multimodal AI Teacher: Integrating Edge Computing and Reasoning Models for Enhanced Student Error Analysis
• OpenVision 2: A Family of Generative Pretrained Visual Encoders for Multimodal Learning
• LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model
• TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Models
• LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compression
• CAD2DMD-SET: Synthetic Generation Tool of Digital Measurement Device CAD Model Datasets for fine-tuning Large Vision-Language Models
• Is this chart lying to me? Automating the detection of misleading visualizations
• GRAFT: GRaPH and Table Reasoning for Textual Alignment -- A Benchmark for Structured Instruction Following and Visual Reasoning
• KRETA: A Benchmark for Korean Reading and Reasoning in Text-Rich VQA Attuned to Diverse Visual Contexts
• Do MLLMs Really Understand the Charts?
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning
• Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation
• VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark
• Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation
• LangVision-LoRA-NAS: Neural Architecture Search for Variable LoRA Rank in Vision Language Models
• Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answering
• Thyme: Think Beyond Images
• Ovis2.5 Technical Report
• MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents
• MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
• BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning
• MoIIE: Mixture of Intra- and Inter-Modality Experts for Large Vision Language Models
• The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Models
• From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text
• Segmenting and Understanding: Region-aware Semantic Attention for Fine-grained Image Quality Assessment with Large Language Models
• InterChart: Benchmarking Visual Reasoning Across Decomposed and Distributed Chart Information
• MolmoAct: Action Reasoning Models that can Reason in Space
• AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning
• DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding
• Effective Training Data Synthesis for Improving MLLM Chart Understanding
• InfoCausalQA:Can Models Perform Non-explicit Causal Reasoning Based on Infographic?
• MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models
• Shuffle-R1: Efficient RL framework for Multimodal Large Language Models via Data-centric Dynamic Shuffle
• LLaVA-RE: Binary Image-Text Relevancy Evaluation with Multimodal Large Language Model
• VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization
• Finding Needles in Images: Can Multimodal LLMs Locate Fine Details?
• VER-Bench: Evaluating MLLMs on Reasoning with Fine-Grained Visual Evidence
• EncQA: Benchmarking Vision-Language Models on Visual Encodings for Charts
• Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?
• VLMQ: Efficient Post-Training Quantization for Large Vision-Language Models via Hessian Augmentation
• ChartCap: Mitigating Hallucination of Dense Chart Captioning
• Evaluating Variance in Visual Question Answering Benchmarks
• ChatVis: Large Language Model Agent for Generating Scientific Visualizations
• VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning
• MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
• MAGE: Multimodal Alignment and Generation Enhancement via Bridging Visual and Semantic Spaces
• METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Models
• AgroBench: Vision-Language Model Benchmark in Agriculture
• Multi-Agent Interactive Question Generation Framework for Long Document Understanding
• Region-based Cluster Discrimination for Visual Representation Learning
• ChartM³: Benchmarking Chart Editing with Multimodal Instructions
• MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents
• SESR-Eval: Dataset for Evaluating LLMs in the Title-Abstract Screening of Systematic Reviews

Related

• ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering
• Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA
• ChartQA-X: Generating Explanations for Visual Chart Reasoning
• RealCQA: Scientific Chart Question Answering as a Test-bed for First-Order Logic
• ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models