2022/06/14 by Anil Palepu, Andrew L. Beam, Palepu, Anil +1
Computer Science · Medicine · #AI in cancer detection #COVID-19 diagnosis using AI #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Radiomics and Machine Learning in Medical Imaging
paper · pdf · doi:10.48550/arxiv.2206.07155
openalex publication_date 2022/06/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Deep learning models trained in a fully supervised manner have been shown to rely on so-called "shortcut" features. Shortcut features are inputs that are associated with the outcome of interest in the training data, but are either no longer associated or not present in testing or deployment settings. Here we provide experiments that show recent self-supervised models trained on images and text provide more robust image representations and reduce the model's reliance on visual shortcut features on a realistic medical imaging example. Additionally, we find that these self-supervised models "forget" shortcut features more quickly than fully supervised ones when fine-tuned on labeled data. Though not a complete solution, our experiments provide compelling evidence that self-supervised models trained on images and text provide some resilience to visual shortcut features.