COLLECT: A Counterfactual Oral Lesion Library for Trustworthy AI Evaluation

2026/07/24 by K.T. Tafti, Z. D’Souza, Amena Ajam +17
Medicine · Computer Science · #Cutaneous Melanoma Detection and Management #AI in cancer detection #Head and Neck Cancer Studies

paper · doi:10.1177/00220345261466240

Abstract

Intraoral mucosal lesions are difficult to diagnose because they vary widely in appearance and cause. Deep-learning models perform well for oral lesion classification, but limited interpretability restricts clinical use. Counterfactual explanations, which modify image features to test model sensitivity, offer a practical way to improve transparency, yet no counterfactual dataset exists for intraoral images. This study presents COLLECT, the Counterfactual Oral Lesion Library for Explainable Concept Testing, a curated counterfactual reference dataset and evaluation framework for intraoral lesion images, along with a workflow for assessing model interpretability. COLLECT includes 600 manually edited counterfactual images across 4 lesion types: aphthous ulcer, geographic tongue, hairy tongue, and oral squamous cell carcinoma. Images were modified along clinically important concepts such as size, color, and opacity. We assessed the interpretability of common convolutional neural networks using true class probability, predictive entropy, testing with concept activation vectors (TCAV), and attribution scores. Counterfactual edits reduced true class probability and increased entropy, with the strongest effects observed for opacity and color changes. TCAV results showed that early network layers were less responsive to concept changes, while later layers depended more on baseline features. Attribution scores confirmed that opacity changes lowered model confidence most, followed by color and size. Some models showed limited response to lesion removal, indicating reliance on background anatomy rather than lesion-specific cues. COLLECT supports the systematic evaluation of computer vision models under clinically grounded counterfactual conditions and provides a resource for improving transparency in the automated diagnosis of oral lesions.

Citations

• Automated Detection of Oral Pre-Cancerous Tongue Lesions Using Deep Learning for Early Diagnosis of Oral Cavity Cancer
• Transparency of deep neural networks for medical image analysis: A review of interpretability methods

Related

• Interpretable Machine Learning for Oral Lesion Diagnosis through Prototypical Instances Identification
• A Novel Approach using CapsNet and Deep Belief Network for Detection and Identification of Oral Leukopenia
• Data-Augmented Multimodal Feature Fusion for Multiclass Visual Recognition of Oral Cancer Lesions
• RAA-MIL: A Novel Framework for Classification of Oral Cytology
• Improving Diagnostic Accuracy for Oral Cancer with inpainting Synthesis Lesions Generated Using Diffusion Models