2024/04/27 by Nadia Saeed, Saeed, Nadia · 1 citation
Arts and Humanities · Health Professions · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Discourse Analysis in Language Studies #FOS: Computer and information sciences #Health Literacy and Information Accessibility #Machine Learning (cs.LG) #Wikis in Education and Collaboration
paper · pdf · doi:10.48550/arxiv.2405.01583
openalex publication_date 2024/04/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The MEDIQA-M3G 2024 challenge necessitates novel solutions for Multilingual & Multimodal Medical Answer Generation in dermatology (wai Yim et al., 2024a). This paper addresses the limitations of traditional methods by proposing a weakly supervised learning approach for open-ended medical question-answering (QA). Our system leverages readily available MEDIQA-M3G images via a VGG16-CNN-SVM model, enabling multilingual (English, Chinese, Spanish) learning of informative skin condition representations. Using pre-trained QA models, we further bridge the gap between visual and textual information through multimodal fusion. This approach tackles complex, open-ended questions even without predefined answer choices. We empower the generation of comprehensive answers by feeding the ViT-CLIP model with multiple responses alongside images. This work advances medical QA research, paving the way for clinical decision support systems and ultimately improving healthcare delivery.