VQA: Visual Question Answering
2015/05/03 by Aishwarya Agrawal, Jiasen Lu, Agrawal, Aishwarya +11 · 507 citations
Computer Science · #Multimodal Machine Learning Applications #Advanced Image and Video Retrieval Techniques #Domain Adaptation and Few-Shot Learning
paper · pdf · doi:10.48550/arxiv.1505.00468
Abstract
We propose the task of free-form and open-ended Visual Question Answering (VQA). Given an image and a natural language question about the image, the task is to provide an accurate natural language answer. Mirroring real-world scenarios, such as helping the visually impaired, both the questions and answers are open-ended. Visual questions selectively target different areas of an image, including background details and underlying context. As a result, a system that succeeds at VQA typically needs a more detailed understanding of the image and complex reasoning than a system producing generic image captions. Moreover, VQA is amenable to automatic evaluation, since many open-ended answers contain only a few words or a closed set of answers that can be provided in a multiple-choice format. We provide a dataset containing ~0.25M images, ~0.76M questions, and ~10M answers (www.visualqa.org), and discuss the information it provides. Numerous baselines and methods for VQA are provided and compared with human performance. Our VQA demo is available on CloudCV (http://cloudcv.org/vqa).
Citations
Cited by
- TOGA: Temporally Grounded Open-Ended Video QA with Weak Supervision
- Instruction-Following Evaluation of Large Vision-Language Models
- Towards Signboard-Oriented Visual Question Answering: ViSignVQA Dataset, Method and Benchmark
- Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
- PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
- A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions
- Group Preference Collapse in Personalized Multimodal Large Language Models
- MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
- Asymmetric Hierarchical Anchoring for Robust Audio-Visual Cross-Modal Generalization
- Toward Intelligent Scene Augmentation for Context-Aware Object Placement and Sponsor-Logo Integration
- TongSIM: A General Platform for Simulating Intelligent Machines
- QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models
- OpenView: Empowering MLLMs with Out-of-view VQA
- Who Can See Through You? Adversarial Shielding Against VLM-Based Attribute Inference Attacks
- Differences That Matter: Auditing Models for Capability Gap Discovery and Rectification
- Scaling Spatial Reasoning in MLLMs through Programmatic Data Synthesis
- Evaluating Large Language Models on Multimodal Chemistry Olympiad Exams
- Enhancing Interpretability for Vision Models via Shapley Value Optimization
- Enhancing Visual Programming for Visual Reasoning via Probabilistic Graphs
- AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
- ViInfographicVQA: A Benchmark for Single and Multi-image Visual Question Answering on Vietnamese Infographics
- Exploring MLLM-Diffusion Information Transfer with MetaCanvas
- Benchmarking the Generality of Vision-Language-Action Models
- Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors
- A Multi-Robot Platform for Robotic Triage Combining Onboard Sensing and Foundation Models
- X-GGM: Graph Generative Modeling for Out-of-Distribution Generalization in Visual Question Answering
- DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoning
- The Role of Entropy in Visual Grounding: Analysis and Optimization
- Stitch and Tell: A Structured Multimodal Data Augmentation Method for Spatial Understanding
- Detector-Empowered Video Large Language Model for Efficient Spatio-Temporal Grounding
- Embodied Referring Expression Comprehension in Human-Robot Interaction
- Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models
- BeLLA: End-to-End Birds Eye View Large Language Assistant for Autonomous Driving
- M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG
- SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding
- Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities
- See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Models
- ChromouVQA: Benchmarking Vision-Language Models under Chromatic Camouflaged Images
- AFRAgent : An Adaptive Feature Renormalization Based High Resolution Aware GUI agent
- Concept-Guided Backdoor Attack on Vision Language Models
- ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering
- Artwork Interpretation with Vision Language Models: A Case Study on Emotions and Emotion Symbols
- Unexplored flaws in multiple-choice VQA evaluations
- WearVQA: A Visual Question Answering Benchmark for Wearables in Egocentric Authentic Real-world scenarios
- CaptionQA: Is Your Caption as Useful as the Image Itself?
- TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering
- Grid-VLP: Revisiting Grid Features for Vision-Language Pre-training
- Cross Domain Evaluation of Multimodal Chain-of-Thought Reasoning of different datasets into the Amazon CoT Framework
- CLASH: A Benchmark for Cross-Modal Contradiction Detection
- BackdoorVLM: A Benchmark for Backdoor Attacks on Vision-Language Models
- Mitigating Long-Tail Bias in HOI Detection via Adaptive Diversity Cache
- RoadSceneVQA: Benchmarking Visual Question Answering in Roadside Perception Systems for Intelligent Transportation System
- VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging
- MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use
- Transformers in Vision: A Survey
- Don't Learn, Ground: A Case for Natural Language Inference with Visual Grounding
- MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models
- Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue
- Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
- Learning Visual Features from Large Weakly Supervised Data
- Semantic Glitch: Agency and Artistry in an Autonomous Pixel Cloud
- Searching in Space and Time: Unified Memory-Action Loops for Open-World Object Retrieval
- SweeperBot: Making 3D Browsing Accessible through View Analysis and Visual Question Answering
- Commonly Uncommon: Semantic Sparsity in Situation Recognition
- Dual-LoRA and Quality-Enhanced Pseudo Replay for Multimodal Continual Food Learning
- Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation
- From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models
- Compositional Generalization for Primitive Substitutions
- ViP-CNN: Visual Phrase Guided Convolutional Neural Network
- Lessons learned in multilingual grounded language learning
- A Diagram Is Worth A Dozen Images
- Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data
- OAD-Promoter: Enhancing Zero-shot VQA using Large Language Models with Object Attribute Description
- Hindsight Distillation Reasoning with Knowledge Encouragement Preference for Knowledge-based Visual Question Answering
- Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
- FaithAct: Faithfulness Planning and Acting in MLLMs
- VideoChain: A Transformer-Based Framework for Multi-hop Video Question Generation
- Cross Modal Fine-Grained Alignment via Granularity-Aware and Region-Uncertain Modeling
- Recurrent Models for Situation Recognition
- TutorialVQA: Question Answering Dataset for Tutorial Videos
- MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning
- Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models
- Unveiling Modality Bias: Automated Sample-Specific Analysis for Multimodal Misinformation Benchmarks
- Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcuts
- IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
- Thought-For-Food: Reasoning Chain Induced Food Visual Question Answering
- An Analysis of Visual Question Answering Algorithms
- Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
- Survey of Recent Advances in Visual Question Answering
- When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought
- Text-VQA Aug: Pipelined Harnessing of Large Multimodal Models for Automated Synthesis
- MM-OPERA: Benchmarking Open-ended Association Reasoning for Large Vision-Language Models
- Masked Diffusion Captioning for Visual Feature Learning
- SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models
- CRAG-MM: Multi-modal Multi-turn Comprehensive RAG Benchmark
- Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation
- Context-aware Captions from Context-agnostic Supervision
- ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding
- Towards Ecologically Valid Research on Language User Interfaces
- Visual Entailment: A Novel Task for Fine-Grained Image Understanding
- Break It Down: A Question Understanding Benchmark
- Recursive Visual Attention in Visual Dialog
- Multimodal Differential Network for Visual Question Generation
- BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and\n Visual Relationship Detection
- Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training
- Challenging Images For Minds and Machines
- Scaling Egocentric Vision: The EPIC-KITCHENS Dataset
- A Restricted Visual Turing Test for Deep Scene and Event Understanding
- A Joint Speaker-Listener-Reinforcer Model for Referring Expressions
- A Focused Dynamic Attention Model for Visual Question Answering
- MMFT-BERT: Multimodal Fusion Transformer with BERT Encodings for Visual\n Question Answering
- Paying Attention to Descriptions Generated by Image Captioning Models
- Video Highlight Prediction Using Audience Chat Reactions
- Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering
- Flickr30k Entities: Collecting Region-to-Phrase Correspondences for\n Richer Image-to-Sentence Models
- Visual Question Answering: A Survey of Methods and Datasets
- Mitigating Modal Imbalance in Multimodal Reasoning
- Evaluating Text-to-Image Matching using Binary Image Selection (BISON)
- Recent Advances in Neural Program Synthesis
- ShapeWorld - A new test methodology for multimodal language understanding
- Enforcing Reasoning in Visual Commonsense Reasoning
- VideoFDB: Evaluating Full-Duplex Vision-Speech Capabilities in Conversational Agents
- Dynamic Memory Networks for Visual and Textual Question Answering
- Answering Questions about Data Visualizations using Efficient Bimodal Fusion
- Image-Question-Answer Synergistic Network for Visual Dialog
- Motion-Appearance Co-Memory Networks for Video Question Answering
- Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation
- REMIND Your Neural Network to Prevent Catastrophic Forgetting
- Grounding Open-Domain Instructions to Automate Web Support Tasks
- A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC
- Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
- A review of uncertainty quantification in deep learning: Techniques, applications and challenges
- WebQA: Multihop and Multimodal QA
- Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning
- Adversarial NLI: A New Benchmark for Natural Language Understanding
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
- Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner
- A Video Is Not Worth a Thousand Words
- MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering
- Windsock is Dancing: Adaptive Multimodal Retrieval-Augmented Generation
- CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication
- OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Models
- Transferable Representation Learning in Vision-and-Language Navigation
- Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidence
- Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context
- Multimedia-Aware Question Answering: A Review of Retrieval and Cross-Modal Reasoning Architectures
- Machine Olfaction and Embedded AI Are Shaping the New Global Sensing Industry
- Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning
- StarBench: A Turn-Based RPG Benchmark for Agentic Multimodal Decision-Making and Information Seeking
- An Interpretable Model for Scene Graph Generation
- Quantifying Multimodal Imbalance: A GMM-Guided Adaptive Loss for Audio-Visual Learning
- Simple Baseline for Visual Question Answering
- Cost Savings from Automatic Quality Assessment of Generated Images
- CaptionFormer: Unified Segmentation, Tracking, and Captioning for Spatio-Temporal Objects
- Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering
- Factor Graph Attention
- ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data
- Visual7W: Grounded Question Answering in Images
- MMED: A Multi-domain and Multi-modality Event Dataset
- IL3D: A Large-Scale Indoor Layout Dataset for LLM-Driven 3D Scene Generation
- VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritage
- Visual Question Answering Using Semantic Information from Image\n Descriptions
- Coupled Degradation Modeling and Fusion: A VLM-Guided Degradation-Coupled Network for Degradation-Aware Infrared and Visible Image Fusion
- Human Uncertainty-Aware Data Selection and Automatic Labeling in Visual Question Answering
- J-ORA: A Framework and Multimodal Dataset for Japanese Object Identification, Reference, Action Prediction in Robot Perception
- Prompt-Guided Spatial Understanding with RGB-D Transformers for Fine-Grained Object Relation Reasoning
- Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQA
- RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering
- Grounding Referring Expressions in Images by Variational Context
- CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization
- VQABQ: Visual Question Answering by Basic Questions
- Boosting Multi-modal Keyphrase Prediction with Dynamic Chain-of-Thought in Vision-Language Models
- To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
- RAVEN: A Dataset for Relational and Analogical Visual rEasoNing
- Multiple interaction learning with question-type prior knowledge for constraining answer search space in visual question answering
- Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
- ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models
- Dataset Reuse: Toward Translating Principles to Practice
- StaR-KVQA: Structured Reasoning Traces for Implicit-Knowledge Visual Question Answering
- Detection and Measurement of Hailstones with Multimodal Large Language Models
- Generation and Comprehension of Unambiguous Object Descriptions
- AiR: Attention with Reasoning Capability
- Scene Graph Generation from Objects, Phrases and Region Captions
- 3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks
- IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance
- V-HUB: A Visual-Centric Humor Understanding Benchmark for Video LLMs
- GroundSight: Augmenting Vision-Language Models with Grounding Information and De-hallucination
- From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
- Skip-It? Theoretical Conditions for Layer Skipping in Vision-Language Models
- VisualOverload: Probing Visual Understanding of VLMs in Really Dense Scenes
- ColLab: A Collaborative Spatial Progressive Data Engine for Referring Expression Comprehension and Generation
- AutoPrune: Each Complexity Deserves a Pruning Policy
- Q-FSRU: Quantum-Augmented Frequency-Spectral For Medical Visual Question Answering
- Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional
- Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflection
- MMPB: It's Time for Multi-Modal Personalization
- Cross-Modality Relevance for Reasoning on Language and Vision
- Visual Relationship Detection using Scene Graphs: A Survey
- Explaining multimodal LLMs via intra-modal token interactions
- RAU: Reference-based Anatomical Understanding with Vision Language Models
- Multilingual Vision-Language Models, A Survey
- How Accurate Are LLMs at Multi-Question Answering on Conversational Transcripts?
- Integrating Object Interaction Self-Attention and GAN-Based Debiasing for Visual Question Answering
- Confidence-guided Refinement Reasoning for Zero-shot Question Answering
- Self-Critical Reasoning for Robust Visual Question Answering
- LXMERT: Learning Cross-Modality Encoder Representations from Transformers
- A Simple Data Augmentation Strategy for Text-in-Image Scientific VQA
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- Unifying Adversarially Robust Model Experts in Vision-Language Models
- Objective-Aligned Direct Answer SFT for Robust Multi-Frame Medical VQA
- MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models
- Sign Language Question Answering: A New Task, Benchmark, and Baseline for Sign Language Understanding
- Distilling Answer Set Programming Theories from Large Language Models
- Leveraging Visual Question Answering for Image-Caption Ranking
- The Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation From Recognition to Reasoning
- Scene Graph Reasoning for Visual Question Answering
- Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
- Multimodal Learning for Hateful Memes Detection
- Self-supervised pre-training and contrastive representation learning for multiple-choice video QA
- From Global to Local: Social Bias Transfer in CLIP
- LCMF: Lightweight Cross-Modality Mambaformer for Embodied Robotics VQA
- Steering Multimodal Large Language Models Decoding for Context-Aware Safety
- Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models
- ColorBlindnessEval: Can Vision-Language Models Pass Color Blindness Tests?
- Memory-QA: Answering Recall Questions Based on Multimodal Memories
- M3ET: Efficient Vision-Language Learning for Robotics based on Multimodal Mamba-Enhanced Transformer
- NeuS-QA: Grounding Long-Form Video Understanding in Temporal Logic and Neuro-Symbolic Reasoning
- Fooling Vision and Language Models Despite Localization and Attention Mechanism
- Aligned Image-Word Representations Improve Inductive Transfer Across\n Vision-Language Tasks
- How Much Can CLIP Benefit Vision-and-Language Tasks?
- When Big Models Train Small Ones: Label-Free Model Parity Alignment for Efficient Visual Question Answering using Small VLMs
- Enhancing Scientific Visual Question Answering via Vision-Caption aware Supervised Fine-Tuning
- A Shortcut-aware Video-QA Benchmark for Physical Understanding via Minimal Video Pairs
- LLM Jailbreak Detection for (Almost) Free!
- A Multi-To-One Interview Paradigm for Efficient MLLM Evaluation
- VCBench: Benchmarking LLMs in Venture Capital
- Structured Attentions for Visual Question Answering
- A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Concepts
- OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contexts
- Pre-Manipulation Alignment Prediction with Parallel Deep State-Space and Transformer Models
- Explain Before You Answer: A Survey on Compositional Visual Reasoning
- Spatial-Temporal Human-Object Interaction Detection
- Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
- CLOSURE: Assessing Systematic Generalization of CLEVR Models
- Query-controllable Video Summarization
- MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapes
- ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language
- Learning Two-Branch Neural Networks for Image-Text Matching Tasks
- Sitatapatra: Blocking the Transfer of Adversarial Samples
- Lost in Embeddings: Information Loss in Vision-Language Models
- MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUs
- Seeing is Not Understanding: A Benchmark on Perception-Cognition Disparities in Large Language Models
- Understanding the Role of Scene Graphs in Visual Question Answering
- Adaptive Token Merging for Efficient Transformer Semantic Communication at the Edge
- Relation-Aware Graph Attention Network for Visual Question Answering
- DVQA: Understanding Data Visualizations via Question Answering
- Unified Multimodal Model as Auto-Encoder
- Retinal Vessel Segmentation under Extreme Low Annotation: A Generative Adversarial Network Approach
- A Knowledge Noise Mitigation Framework for Knowledge-based Visual Question Answering
- Can Vision-Language Models Solve Visual Math Equations?
- Generalized User-Oriented Image Semantic Coding Empowered by Large Vision-Language Model
- Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles
- BcQLM: Efficient Vision-Language Understanding with Distilled Q-Gated Cross-Modal Fusion
- PlantExpertVQA: A Visual Question Answering Dataset for Benchmarking Vision-Language Models in Plant Science
- Interpretable Physics Reasoning and Performance Taxonomy in Vision-Language Models
- SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation
- Deep Image Retrieval: Learning global representations for image search
- Answering Visual What-If Questions: From Actions to Predicted Scene Descriptions
- Parse Graph-Based Visual-Language Interaction for Human Pose Estimation
- Prototype-Aware Multimodal Alignment for Open-Vocabulary Visual Grounding
- Faithful Multimodal Explanation for Visual Question Answering
- SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering
- Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs
- Language-Driven Object-Oriented Two-Stage Method for Scene Graph Anticipation
- Dual-Domain Perspective on Degradation-Aware Fusion: A VLM-Guided Robust Infrared and Visible Image Fusion Framework
- Can VLMs Recall Factual Associations From Visual References?
- The Telephone Game: Evaluating Semantic Drift in Unified Models
- Quantifying and Alleviating the Language Prior Problem in Visual Question Answering
- Promptception: How Sensitive Are Large Multimodal Models to Prompts?
- Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback
- Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
- ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- Grounded Agreement Games: Emphasizing Conversational Grounding in Visual Dialogue Settings
- Hierarchical Vision-Language Reasoning for Multimodal Multiple-Choice Question Answering
- Going Beneath the Surface: Evaluating Image Captioning for Grammaticality, Truthfulness and Diversity
- GEM: A General Evaluation Benchmark for Multimodal Tasks
- Street-Level Geolocalization Using Multimodal Large Language Models and Retrieval-Augmented Generation
- Person Search with Natural Language Description
- The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives
- No More Sibling Rivalry: Debiasing Human-Object Interaction Detection
- Understanding Art through Multi-Modal Retrieval in Paintings
- ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answering
- Dual Attention Networks for Multimodal Reasoning and Matching
- Bangla-Bayanno: A 52K-Pair Bengali Visual Question Answering Dataset with LLM-Assisted Translation Refinement
- Scalable Object Detection in the Car Interior With Vision Foundation Models
- DeepThink3D: Enhancing Large Language Models with Programmatic Reasoning in Complex 3D Situated Reasoning Tasks
- Measuring and Improving Consistency in Pretrained Language Models
- Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum
- Extracting Information from Scientific Literature via Visual Table Question Answering Models
- DIO: Refining Mutual Information and Causal Chain to Enhance Machine Abstract Reasoning Ability
- SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Models
- ManyModalQA: Modality Disambiguation and QA over Diverse Inputs
- Emergence of Compositional Language with Deep Generational Transmission
- EcomMMMU: Strategic Utilization of Visuals for Robust Multimodal E-commerce Models
- Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition
- Large Graph Exploration via Subgraph Discovery and Decomposition
- Mitigating Cross-Image Information Leakage in LVLMs for Multi-Image Tasks
- Mitigating Easy Option Bias in Multiple-Choice Question Answering
- SPANER: Shared Prompt Aligner for Multimodal Semantic Representation
- Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
- Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation
- Checkmate: interpretable and explainable RSVQA is the endgame
- RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts
- Exploring Multimodal AI Reasoning for Meteorological Forecasting from Skew-T Diagrams
- Adversarial Attacks on VQA-NLE: Exposing and Alleviating Inconsistencies in Visual Question Answering Explanations
- Learning to Globally Edit Images with Textual Description
- Loss re-scaling VQA: Revisiting the LanguagePrior Problem from a Class-imbalance View
- Q-FSRU: Quantum-Augmented Frequency-Spectral Fusion for Medical Visual Question Answering
- Dataset Creation for Visual Entailment using Generative AI
- Using Thought-Provoking Children's Questions to Drive Artificial\n Intelligence Research
- ORBIT: An Object Property Reasoning Benchmark for Visual Inference Tasks
- Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies
- A Curriculum Learning Approach to Reinforcement Learning: Leveraging RAG for Multimodal Question Answering
- Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation
- LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit
- VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models
- SemVLP: Vision-Language Pre-training by Aligning Semantics at Multiple Levels
- Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions
- Text-conditioned State Space Model For Domain-generalized Change Detection Visual Question Answering
- StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI
- Vision Generalist Model: A Survey
- Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning
- Story Ribbons: Reimagining Storyline Visualizations with Large Language Models
- AGIC: Attention-Guided Image Captioning to Improve Caption Relevance
- Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
- AdaptInfer: Adaptive Token Pruning for Vision-Language Model Inference with Dynamical Text Guidance
- mKG-RAG: Multimodal Knowledge Graph-Enhanced RAG for Visual Question Answering
- VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization
- X-SAM: From Segment Anything to Any Segmentation
- Do Recommender Systems Really Leverage Multimodal Content? A Comprehensive Analysis on Multimodal Representations for Recommendation
- Human-centric Relation Segmentation: Dataset and Solution
- Decoding the Multimodal Maze: A Systematic Review on the Adoption of Explainability in Multimodal Attention-based Models
- OmniPlay: Benchmarking Omni-Modal Models on Omni-Modal Game Playing
- Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting
- Leveraging Visual Question Answering to Improve Text-to-Image Synthesis
- VQA support to Arabic Language Learning Educational Tool
- Facilitating Visual Media Exploration for Blind and Low Vision Users through AI-Powered Interactive Storytelling
- ReasonAct: Progressive Training for Fine-Grained Video Reasoning in Small Models
- Artifacts and Attention Sinks: Structured Approximations for Efficient Vision Transformers
- MultiSHAP: A Shapley-Based Framework for Explaining Cross-Modal Interactions in Multimodal AI Models
- Oedipus and the Sphinx: Benchmarking and Improving Visual Language Models for Complex Graphic Reasoning
- Relation Networks for Object Detection
- ART: Adaptive Relation Tuning for Generalized Relation Prediction
- Saliency-Guided Attention Network for Image-Sentence Matching
- Benchmark Visual Question Answer Models by using Focus Map
- Describing Unseen Videos via Multi-Modal Cooperative Dialog Agents
- HW-MLVQA: Elucidating Multilingual Handwritten Document Understanding with a Comprehensive VQA Benchmark
- Modality-Aware Feature Matching: A Comprehensive Review of Single- and Cross-Modality Techniques
- CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset
- On the Reliability of Vision-Language Models Under Adversarial Frequency-Domain Perturbations
- DeltaVLM: Interactive Remote Sensing Image Change Analysis via Instruction-guided Difference Perception
- VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning
- Entropy-Enhanced Multimodal Attention Model for Scene-Aware Dialogue Generation
- Analyzing the Sensitivity of Vision Language Models in Visual Question Answering
- Think Visually: Question Answering through Virtual Imagery
- On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey
- Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning
- HUMBO: Bridging Response Generation and Facial Expression Synthesis
- Detecting Visual Information Manipulation Attacks in Augmented Reality: A Multimodal Semantic Reasoning Approach
- Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection
- VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions
- Learning Language-Visual Embedding for Movie Understanding with Natural-Language
- Causality-aligned Prompt Learning via Diffusion-based Counterfactual Generation
- What is needed for simple spatial language capabilities in VQA?
- Towards a Robust Deep Neural Network in Texts: A Survey
- Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog
- A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets
- Scene Graph Reasoning with Prior Visual Relationship for Visual Question Answering
- Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
- True Multimodal In-Context Learning Needs Attention to the Visual Context
- RL-CSDia: Representation Learning of Computer Science Diagrams
- Interactive Text2Pickup Network for Natural Language based Human-Robot\n Collaboration
- Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasets
- Augmented Vision-Language Models: A Systematic Review
- Action2Vec: A Crossmodal Embedding Approach to Action Learning
- MTLE: A Multitask Learning Encoder of Visual Feature Representations for\n Video and Movie Description
- Are VQA Systems RAD? Measuring Robustness to Augmented Data with Focused Interventions
- Towards Video Thinking Test: A Holistic Benchmark for Advanced Video Reasoning and Understanding
- The Wisdom of MaSSeS: Majority, Subjectivity, and Semantic Similarity in the Evaluation of VQA
- Med-GRIM: Enhanced Zero-Shot Medical VQA using prompt-embedded Multimodal Graph RAG
- Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering
- Pay Attention to Those Sets! Learning Quantification from Images
- Pairwise Relations Discriminator for Unsupervised Raven's Progressive Matrices
- MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification
- Visual Reasoning with Natural Language
- Zero-shot task adaptation by homoiconic meta-mapping
- Check It Again: Progressive Visual Question Answering via Visual Entailment
- Multimodal AI for Gastrointestinal Diagnostics: Tackling VQA in MEDVQA-GI 2025
- Modeling Image Virality with Pairwise Spatial Transformer Networks
- Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association
- Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations
- Robustness Analysis of Visual QA Models by Basic Questions
- Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark
- Speech-Based Visual Question Answering
- Variational Inference for Learning Representations of Natural Language Edits
- A Picture May Be Worth a Hundred Words for Visual Question Answering
- XAlgo: a Design Probe of Explaining Algorithms' Internal States via Question-Answering
- VisionTrap: Unanswerable Questions On Visual Data
- Differential-informed Sample Selection Accelerates Multimodal Contrastive Learning
- Language bias in Visual Question Answering: A Survey and Taxonomy
- Pixels to Principles: Probing Intuitive Physics Understanding in Multimodal Language Models
- Turning Internal Gap into Self-Improvement: Promoting the Generation-Understanding Unification in MLLMs
- Building a Large-scale Multimodal Knowledge Base System for Answering Visual Queries
- Natural Language Guided Visual Relationship Detection
- COREVQA: A Crowd Observation and Reasoning Entailment Visual Question Answering Benchmark
- Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It
- Describe Anything Model for Visual Question Answering on Text-rich Images
- AutoVDC: Automated Vision Data Cleaning Using Vision-Language Models
- Modulating early visual processing by language
- Outside Knowledge Conversational Video (OKCV) Dataset -- Dialoguing over Videos
- ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
- The Color of the Cat is Gray: 1 Million Full-Sentences Visual Question Answering (FSVQA)
- Natural Language Understanding with Distributed Representation
- Building a Video-and-Language Dataset with Human Actions for Multimodal Logical Inference
- Composing Text and Image for Image Retrieval - An Empirical Odyssey
- DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for Parameter-Efficient Video-Text Retrieval
- TGIF: A New Dataset and Benchmark on Animated GIF Description
- Incorporating External Knowledge to Answer Open-Domain Visual Questions with Dynamic Memory Networks
- Recent Advances and Trends in Multimodal Deep Learning: A Review
- Conversational AI Systems for Social Good: Opportunities and Challenges
- Question-Conditioned Counterfactual Image Generation for VQA
- Making History Matter: History-Advantage Sequence Training for Visual Dialog
- Dense Captioning with Joint Inference and Visual Context
- Pre-Trained Models: Past, Present and Future
- Multi-level Multimodal Common Semantic Space for Image-Phrase Grounding
- Conceptualizing Multi-scale Wavelet Attention and Ray-based Encoding for Human-Object Interaction Detection
- OOWL500: Overcoming Dataset Collection Bias in the Wild
- Identity-Aware Textual-Visual Matching with Latent Co-attention
- Learning Cooperative Visual Dialog Agents with Deep Reinforcement\n Learning
- CoralVQA: A Large-Scale Visual Question Answering Dataset for Coral Reef Image Understanding
- Multimodal Dialogue State Tracking By QA Approach with Data Augmentation
- Visual Question Answering with Memory-Augmented Networks
- Multilingual Multimodal Software Developer for Code Generation
- Large Multi-modal Model Cartographic Map Comprehension for Textual Locality Georeferencing
- Impact of Pretraining Word Co-occurrence on Compositional Generalization in Multimodal Models
- FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representations
- LinguaMark: Do Multimodal Models Speak Fairly? A Benchmark-Based Evaluation
- Video Fill in the Blank with Merging LSTMs
- Explainable Artificial Intelligence in Biomedical Image Analysis: A Comprehensive Survey
- FACap: A Large-scale Fashion Dataset for Fine-grained Composed Image Retrieval
- Visual Referring Expression Recognition: What Do Systems Actually Learn?
- A System for Automated Image Editing from Natural Language Commands
- Computed Tomography Visual Question Answering with Cross-modal Feature Graphing
- Answer Them All! Toward Universal Visual Question Answering Models
- Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
- MolVision: Molecular Property Prediction with Vision Language Models
- Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
- Unsupervised Keyword Extraction for Full-sentence VQA
- Revisiting Deep Architectures for Head Motion Prediction in 360° Videos
- ESTR-CoT: Towards Explainable and Accurate Event Stream based Scene Text Recognition with Chain-of-Thought Reasoning
- Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentation
- End-to-end optimization of goal-driven and visually grounded dialogue systems Harm de Vries
- A Revised Generative Evaluation of Visual Dialogue
- From Answers to Rationales: Self-Aligning Multimodal Reasoning with Answer-Oriented Chain-of-Thought
- PAC Bench: Do Foundation Models Understand Prerequisites for Executing Manipulation Policies?
- MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI
- Test-Time Consistency in Vision Language Models
- Trying Bilinear Pooling in Video-QA
- What is More Likely to Happen Next? Video-and-Language Future Event Prediction
- Multimodal Misinformation Detection Using Early Fusion of Linguistic, Visual, and Social Features
- Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
- OmniEval: A Benchmark for Evaluating Omni-modal Models with Visual, Auditory, and Textual Inputs
- FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering
- Seeing is Believing? Mitigating OCR Hallucinations in Multimodal Large Language Models
- ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset
- Visual Entailment Task for Visually-Grounded Language Learning
- Supervised Multimodal Bitransformers for Classifying Images and Text
- ToSA: Token Merging with Spatial Awareness
- Question Answering is a Format; When is it Useful?
- ReFrame: Rectification Framework for Image Explaining Architectures
- DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Models
- Cause-Effect Driven Optimization for Robust Medical Visual Question Answering with Language Biases
- Convolutional Neural Networks for Aerial Multi-Label Pedestrian Detection
- Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game
- MEXA: Towards General Multimodal Reasoning with Dynamic Multi-Expert Aggregation
- Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources
- MUCAR: Benchmarking Multilingual Cross-Modal Ambiguity Resolution for Multimodal Large Language Models
- LaVi: Efficient Large Vision-Language Models via Internal Feature Modulation
- GeoGuess: Multimodal Reasoning based on Hierarchy of Visual Information in Street View
- Privacy-Shielded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
- Improving Visual Question Answering by Referring to Generated Paragraph Captions
- ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM
- FinLMM-R1: Enhancing Financial Reasoning in LMM through Scalable Data and Reward Design
- CompGuessWhat?!: A Multi-task Evaluation Framework for Grounded Language\n Learning
- LOP: Learning Optimal Pruning for Efficient On-Demand MLLMs Scaling
- CliniDial: A Naturally Occurring Multimodal Dialogue Dataset for Team Reflection in Action During Clinical Operation
- InverTune: Removing Backdoors from Multimodal Contrastive Learning Models via Trigger Inversion and Activation Tuning
- EasyARC: Evaluating Vision Language Models on True Visual Reasoning
- Aligning MLLM Benchmark With Human Preferences via Structural Equation Modeling
- On the Effectiveness of Integration Methods for Multimodal Dialogue Response Retrieval
- MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space
- Multimodal Machine Learning: A Survey and Taxonomy
- Explicit Knowledge-based Reasoning for Visual Question Answering
- FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
- Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs
- Athena: Enhancing Multimodal Reasoning with Data-efficient Process Reward Models
- Revisit What You See: Disclose Language Prior in Vision Tokens for LVLM Decoding
Related