2021/11/12 by Andréanne Lemay, Katharina Hoebel, Lemay, Andreanne +13
Computer Science · Medicine · #AI in cancer detection #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Machine Learning in Healthcare #Radiomics and Machine Learning in Medical Imaging #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2111.06754
openalex publication_date 2021/11/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The integration of artificial intelligence into clinical workflows requires reliable and robust models. Among the main features of robustness is repeatability. Much attention is given to classification performance without assessing the model repeatability, leading to the development of models that turn out to be unusable in practice. In this work, we evaluate the repeatability of four model types on images from the same patient that were acquired during the same visit. We study the performance of binary, multi-class, ordinal, and regression models on three medical image analysis tasks: cervical cancer screening, breast density estimation, and retinopathy of prematurity classification. Moreover, we assess the impact of sampling Monte Carlo dropout predictions at test time on classification performance and repeatability. Leveraging Monte Carlo predictions significantly increased repeatability for all tasks on the binary, multi-class, and ordinal models leading to an average reduction of the 95% limits of agreement by 17% points.