RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflows

2025/09/24 by Kai Zhang, Zhang, Kai, Jangwon Kim +8
Computer Science · Decision Sciences · #Topic Modeling #Semantic Web and Ontologies #Scientific Computing and Data Management

paper · pdf · doi:10.48550/arxiv.2509.20490

Abstract

Agentic systems offer a potential path to solve complex clinical tasks through collaboration among specialized agents, augmented by tool use and external knowledge bases. Nevertheless, for chest X-ray (CXR) interpretation, prevailing methods remain limited: (i) reasoning is frequently neither clinically interpretable nor aligned with guidelines, reflecting mere aggregation of tool outputs; (ii) multimodal evidence is insufficiently fused, yielding text-only rationales that are not visually grounded; and (iii) systems rarely detect or resolve cross-tool inconsistencies and provide no principled verification mechanisms. To bridge the above gaps, we present RadAgents, a multi-agent framework that couples clinical priors with task-aware multimodal reasoning and encodes a radiologist-style workflow into a modular, auditable pipeline. In addition, we integrate grounding and multimodal retrieval-augmentation to verify and resolve context conflicts, resulting in outputs that are more reliable, transparent, and consistent with clinical practice.

Citations

• Qwen3-VL Technical Report
• EditGRPO: Reinforcement Learning with Post-Rollout Edits for Clinically Accurate Chest X-Ray Report Generation
• MedGemma 1.5 Technical Report
• RadFabric: Agentic AI System with Reasoning Capability for Radiology
• Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning
• CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays
• More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
• Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey
• Reducing Hallucinations of Medical Multimodal Large Language Models with Visual Retrieval-Augmented Generation
• OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning
• MedRAX: Medical Reasoning Agent for Chest X-ray
• VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge
• GPT-4o System Card
• MMedAgent: Learning to Use Medical Tools with Multi-modal Agent
• RaTEScore: A Metric for Radiology Report Generation
• MAIRA-2: Grounded Radiology Report Generation
• CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formats
• AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments
• MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark
• Towards Generalist Biomedical AI
• Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting
• Lost in the Middle: How Language Models Use Long Contexts
• Accurate Fine-Grained Segmentation of Human Anatomy in Radiographs via Volumetric Pseudo-Labeling
• Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
• GPT-4 Technical Report
• Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing
• Distributing Accountability, Not Capability: Phase Separation and the LLM Workflow Quadrant in Autonomous AI Agent Architectures
• TorchXRayVision: A library of chest X-ray datasets and models
• RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
• SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering
• CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT
• MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs
• Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs
• A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation

Related

• RadFabric: Agentic AI System with Reasoning Capability for Radiology
• CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretation
• Interpreting Chest X-rays Like a Radiologist: A Benchmark with Clinical Reasoning
• CXR-Agent: Vision-language models for chest X-ray interpretation with uncertainty aware radiology reporting
• Reasoning Visual Language Model for Chest X-Ray Analysis