Holistic Artificial Intelligence in Medicine; improved performance and explainability

2025/06/30 by Georgios Margaritis, Petridis, Periklis, Margaritis, Georgios +4
Computer Science · Medicine · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2507.00205

openalex publication_date 2025/06/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

With the increasing interest in deploying Artificial Intelligence in medicine, we previously introduced HAIM (Holistic AI in Medicine), a framework that fuses multimodal data to solve downstream clinical tasks. However, HAIM uses data in a task-agnostic manner and lacks explainability. To address these limitations, we introduce xHAIM (Explainable HAIM), a novel framework leveraging Generative AI to enhance both prediction and explainability through four structured steps: (1) automatically identifying task-relevant patient data across modalities, (2) generating comprehensive patient summaries, (3) using these summaries for improved predictive modeling, and (4) providing clinical explanations by linking predictions to patient-specific medical knowledge. Evaluated on the HAIM-MIMIC-MM dataset, xHAIM improves average AUC from 79.9% to 90.3% across chest pathology and operative tasks. Importantly, xHAIM transforms AI from a black-box predictor into an explainable decision support system, enabling clinicians to interactively trace predictions back to relevant patient data, bridging AI advancements with clinical utility.

Citations

• Qwen3 Technical Report
• Medical Hallucinations in Foundation Models and Their Impact on Healthcare
• Qwen2.5-VL Technical Report
• ClinicalBench: Can LLMs Beat Traditional ML Models in Clinical Prediction?
• A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challenges
• Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphs
• Benchmarking Retrieval-Augmented Generation for Medicine
• MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records
• Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine
• Large language models in medicine
• A Survey on Evaluation of Large Language Models
• A Survey on Evaluation of Large Language Models
• Lost in the Middle: How Language Models Use Long Contexts
• EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models
• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
• Large Language Models Encode Clinical Knowledge
• Large language models encode clinical knowledge
• Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation
• Language Models are Few-Shot Learners
• The future of digital health with federated learning
• A Survey on Bias and Fairness in Machine Learning
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

Related

• Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond
• Explainable AI applications in the Medical Domain: a systematic review
• Explainable AI meets Healthcare: A Study on Heart Disease Dataset
• Explainable Artificial Intelligence for Medical Applications: A Review
• Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration