Barriers in Integrating Medical Visual Question Answering into Radiology Workflows: A Scoping Review and Clinicians' Insights

2025/07/09 by Chaklam Silpasuwanchai, Mishra, Deepali, Silpasuwanchai, Chaklam +6
Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2507.08036

openalex publication_date 2025/07/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Medical Visual Question Answering (MedVQA) is a promising tool to assist radiologists by automating medical image interpretation through question answering. Despite advances in models and datasets, MedVQA's integration into clinical workflows remains limited. This study systematically reviews 68 publications (2018-2024) and surveys 50 clinicians from India and Thailand to examine MedVQA's practical utility, challenges, and gaps. Following the Arksey and O'Malley scoping review framework, we used a two-pronged approach: (1) reviewing studies to identify key concepts, advancements, and research gaps in radiology workflows, and (2) surveying clinicians to capture their perspectives on MedVQA's clinical relevance. Our review reveals that nearly 60% of QA pairs are non-diagnostic and lack clinical relevance. Most datasets and models do not support multi-view, multi-resolution imaging, EHR integration, or domain knowledge, features essential for clinical diagnosis. Furthermore, there is a clear mismatch between current evaluation metrics and clinical needs. The clinician survey confirms this disconnect: only 29.8% consider MedVQA systems highly useful. Key concerns include the absence of patient history or domain knowledge (87.2%), preference for manually curated datasets (51.1%), and the need for multi-view image support (78.7%). Additionally, 66% favor models focused on specific anatomical regions, and 89.4% prefer dialogue-based interactive systems. While MedVQA shows strong potential, challenges such as limited multimodal analysis, lack of patient context, and misaligned evaluation approaches must be addressed for effective clinical integration.

Citations

• M2I2: Learning Efficient Multi-Agent Communication via Masked State Modeling and Intention Inference
• MISS: A Generative Pretraining and Finetuning Approach for Med-VQA
• Radiology-Aware Model-Based Evaluation Metric for Report Generation
• EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images
• Med-Flamingo: a Multimodal Medical Few-shot Learner
• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
• Q2ATransformer: Improving Medical VQA via an Answer Querying Decoder
• PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents
• BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs
• MedCLIP: Contrastive Learning from Unpaired Medical Images and Text
• Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training
• Multiple Meta-model Quantifying for Medical Visual Question Answering
• Cross-Modal Self-Attention with Multi-Task Pre-Training for Medical Visual Question Answering
• MMBERT: Multimodal BERT Pretraining for Improved Medical VQA
• SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering
• Hierarchical Deep Multi-modal Network for Medical Visual Question Answering
• CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT
• Overcoming Data Limitation in Medical Visual Question Answering

Related

• Free Form Medical Visual Question Answering in Radiology
• Adapting Lightweight Vision Language Models for Radiological Visual Question Answering
• RadImageNet-VQA: A Large-Scale CT and MRI Dataset for Radiologic Visual Question Answering
• MedThink: Explaining Medical Visual Question Answering via Multimodal Decision-Making Rationale
• Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answering