LLM-Based Visual Explanation Evaluation Framework for Assessing the Explainability of Facial Skin Disease Classification Models

2026/06/15 by Gyuyeon Na
#cs.CV

paper · pdf

Abstract

This study proposes a domain-specific LLM-based Visual Explanation Evaluation Framework for assessing visual attention explanations in facial skin disease diagnosis. While previous studies have primarily focused on improving classification performance, relatively few studies have systematically examined whether visual explanations are grounded in clinically relevant lesion regions. In this study, an image-driven visual attention generation algorithm was developed to produce lesion-focused attention maps from facial skin disease images. Unlike conventional gradient-based explainability methods, the proposed approach combines color saliency, facial spatial priors, Gaussian smoothing, and attention overlay visualization to generate clinically interpretable attention maps. Furthermore, an LLM-as-a-Judge evaluation framework was designed using GPT-5.5, Gemini 3.5 Flash, and Claude Sonnet 4.6 to assess the generated visual explanations from the perspectives of lesion localization and explanation trustworthiness.

Citations

• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Deep Residual Learning for Image Recognition

Related

• Are Explanations Helpful? A Comparative Analysis of Explainability Methods in Skin Lesion Classifiers
• VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosis
• Explainable Deep Image Classifiers for Skin Lesion Diagnosis
• MelanomaNet: Explainable Deep Learning for Skin Lesion Classification
• Fine-tuning of explainable CNNs for skin lesion classification based on dermatologists' feedback towards increasing trust