Toward Effective Reinforcement Learning Fine-Tuning for Medical VQA in Vision-Language Models

2025/05/20 by Wenhui Zhu, Zhu, Wenhui, Xuanzhao Dong +15 · 5 citations
Neuroscience · #Artificial Intelligence (cs.AI) #Brain Tumor Detection and Classification #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2505.13973

openalex publication_date 2025/05/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recently, reinforcement learning (RL)-based tuning has shifted the trajectory of Multimodal Large Language Models (MLLMs), particularly following the introduction of Group Relative Policy Optimization (GRPO). However, directly applying it to medical tasks remains challenging for achieving clinically grounded model behavior. Motivated by the need to align model response with clinical expectations, we investigate four critical dimensions that affect the effectiveness of RL-based tuning in medical visual question answering (VQA): base model initialization strategy, the role of medical semantic alignment, the impact of length-based rewards on long-chain reasoning, and the influence of bias. We conduct extensive experiments to analyze these factors for medical MLLMs, providing new insights into how models are domain-specifically fine-tuned. Additionally, our results also demonstrate that GRPO-based RL tuning consistently outperforms standard supervised fine-tuning (SFT) in both accuracy and reasoning quality.

Cited by

• CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency
• ADHint: Adaptive Hints with Difficulty Priors for Reinforcement Learning
• Reinforcement Learning for Clinical Reasoning: Aligning LLMs with ACR Imaging Appropriateness Criteria
• Patho-AgenticRAG: Towards Multimodal Agentic Retrieval-Augmented Generation for Pathology VLMs via Reinforcement Learning
• LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding

Related

• Adaptive Reinforcement for Open-ended Medical Reasoning via Semantic-Guided Reward Collapse Mitigation
• CAPO: Reinforcing Consistent Reasoning in Medical Decision-Making
• Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models
• The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models
• Improving Medical Reasoning with Curriculum-Aware Reinforcement Learning