Weakly Supervised Concept Learning with Class-Level Priors for Interpretable Medical Diagnosis

2025/11/03 by Md. Nahiduzzaman, Steven Korevaar, Nahiduzzaman, Md +5
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2511.01131

openalex publication_date 2025/11/03 · openalex created_date 2025/11/06 · openalex updated_date 2026/07/28

Abstract

Human-interpretable predictions are essential for deploying AI in medical imaging, yet most interpretable-by-design (IBD) frameworks require concept annotations for training data, which are costly and impractical to obtain in clinical contexts. Recent attempts to bypass annotation, such as zero-shot vision-language models or concept-generation frameworks, struggle to capture domain-specific medical features, leading to poor reliability. In this paper, we propose a novel Prior-guided Concept Predictor (PCP), a weakly supervised framework that enables concept answer prediction without explicit supervision or reliance on language models. PCP leverages class-level concept priors as weak supervision and incorporates a refinement mechanism with KL divergence and entropy regularization to align predictions with clinical reasoning. Experiments on PH2 (dermoscopy) and WBCatt (hematology) show that PCP improves concept-level F1-score by over 33% compared to zero-shot baselines, while delivering competitive classification performance on four medical datasets (PH2, WBCatt, HAM10000, and CXR4) relative to fully supervised concept bottleneck models (CBMs) and V-IP.

Citations

• An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training
• CBVLM: Training-free Explainable Concept-based Large Vision Language Models for Medical Image Classification
• V2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer
• MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment
• Learning Interpretable Queries for Explainable Image Classification with Information Pursuit
• WBCAtt: A White Blood Cell Dataset Annotated with Detailed Morphological Attributes
• Sigmoid Loss for Language Image Pre-Training
• BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs
• Variational Information Pursuit for Interpretable Predictions
• Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification
• Post-hoc Concept Bottleneck Models
• Learning Transferable Visual Models From Natural Language Supervision
• Concept Bottleneck Models

Related

• Integrating Clinical Knowledge into Concept Bottleneck Models
• CBVLM: Training-free Explainable Concept-based Large Vision Language Models for Medical Image Classification
• Training-free Test-time Improvement for Explainable Medical Image Classification
• Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis
• Evidential Concept Embedding Models: Towards Reliable Concept Explanations for Skin Disease Diagnosis