BiomedXPro: Prompt Optimization for Explainable Diagnosis with Biomedical Vision Language Models

2025/10/17 by Silva, Kaushitha, Mansitha Eashwara, Sanduni Ubayasiri +6
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning in Healthcare #Multimodal Machine Learning Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.2510.15866

openalex publication_date 2025/10/17 · openalex created_date 2025/10/21 · openalex updated_date 2026/07/28

Abstract

The clinical adoption of biomedical vision-language models is hindered by prompt optimization techniques that produce either uninterpretable latent vectors or single textual prompts. This lack of transparency and failure to capture the multi-faceted nature of clinical diagnosis, which relies on integrating diverse observations, limits their trustworthiness in high-stakes settings. To address this, we introduce BiomedXPro, an evolutionary framework that leverages a large language model as both a biomedical knowledge extractor and an adaptive optimizer to automatically generate a diverse ensemble of interpretable, natural-language prompt pairs for disease diagnosis. Experiments on multiple biomedical benchmarks show that BiomedXPro consistently outperforms state-of-the-art prompt-tuning methods, particularly in data-scarce few-shot settings. Furthermore, our analysis demonstrates a strong semantic alignment between the discovered prompts and statistically significant clinical features, grounding the model's performance in verifiable concepts. By producing a diverse ensemble of interpretable prompts, BiomedXPro provides a verifiable basis for model predictions, representing a critical step toward the development of more trustworthy and clinically-aligned AI systems.

Citations

• BiomedCoOp: Learning to Prompt for Biomedical Vision-Language Models
• XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization
• Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
• Language Models as Black-Box Optimizers for Vision-Language Models
• WBCAtt: A White Blood Cell Dataset Annotated with Detailed Morphological Attributes
• Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis
• BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs
• MedCLIP: Contrastive Learning from Unpaired Medical Images and Text
• Conditional Prompt Learning for Vision-Language Models
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Learning to Prompt for Vision-Language Models
• Learning Transferable Visual Models From Natural Language Supervision
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Related

• BiomedCoOp: Learning to Prompt for Biomedical Vision-Language Models
• Biomed-DPT: Dual Modality Prompt Tuning for Biomedical Vision-Language Models
• Prompt Triage: Structured Optimization Enhances Vision-Language Model Performance on Medical Imaging Benchmarks
• XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization
• Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?