Teaching AI Stepwise Diagnostic Reasoning with Report-Guided Chain-of-Thought Learning

2025/09/08 by Yu Luo, Luo, Yihong, Wenwu He +5
Computer Science · #Artificial Intelligence (cs.AI) #Domain (mathematical analysis) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Inference #Medical diagnosis #Narrative #Online Learning and Analytics #Reinforcement learning #Scalability

paper · pdf · doi:10.48550/arxiv.2509.06409

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/09/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

This study presents DiagCoT, a multi-stage framework that applies supervised fine-tuning to general-purpose vision-language models (VLMs) to emulate radiologists' stepwise diagnostic reasoning using only free-text reports. DiagCoT combines contrastive image-report tuning for domain alignment, chain-of-thought supervision to capture inferential logic, and reinforcement tuning with clinical reward signals to enhance factual accuracy and fluency. On the MIMIC-CXR benchmark, DiagCoT improved zero-shot disease classification AUC from 0.52 to 0.76 (absolute gain of 0.24), pathology grounding mIoU from 0.08 to 0.31 (absolute gain of 0.23), and report generation BLEU from 0.11 to 0.33 (absolute gain of 0.22). It outperformed state-of-the-art models including LLaVA-Med and CXR-LLAVA on long-tailed diseases and external datasets. By converting unstructured clinical narratives into structured supervision, DiagCoT offers a scalable approach for developing interpretable and diagnostically competent AI systems for radiology.

Citations

• ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification
• Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models
• Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
• Visual-RFT: Visual Reinforcement Fine-Tuning
• MoColl: Agent-Based Specific and General Model Collaboration for Image Captioning
• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
• DeepSeek-VL: Towards Real-World Vision-Language Understanding
• CogVLM: Visual Expert for Pretrained Language Models
• CXR-LLAVA: a multimodal large language model for interpreting chest X-ray images
• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
• Visual Instruction Tuning
• Cross-modal Prototype Driven Network for Radiology Report Generation
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment
• Learning Transferable Visual Models From Natural Language Supervision
• Generating Radiology Reports via Memory-driven Transformer
• CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
• MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs
• Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
• Proximal Policy Optimization Algorithms
• Deep reinforcement learning from human preferences
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Deep Visual-Semantic Alignments for Generating Image Descriptions
• CIDEr: Consensus-based Image Description Evaluation
• Long-term Recurrent Convolutional Networks for Visual Recognition and Description
• A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation

Related

• Radiology's Last Exam (RadLE): Benchmarking Frontier Multimodal AI Against Human Experts and a Taxonomy of Visual Reasoning Errors in Radiology
• Bridging human and machine intelligence: Reverse-engineering radiologist intentions for clinical trust and adoption
• CopilotCAD: Empowering Radiologists with Report Completion Models and Quantitative Evidence from Medical Image Foundation Models
• Reason Like a Radiologist: Chain-of-Thought and Reinforcement Learning for Verifiable Report Generation
• ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification