A Conformal Prediction Score that is Robust to Label Noise

2024/05/04 by Penso, Coby, Goldberger, Jacob · 1 citation
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2405.02648

Abstract

Conformal Prediction (CP) quantifies network uncertainty by building a small prediction set with a pre-defined probability that the correct class is within this set. In this study we tackle the problem of CP calibration based on a validation set with noisy labels. We introduce a conformal score that is robust to label noise. The noise-free conformal score is estimated using the noisy labeled data and the noise level. In the test phase the noise-free score is used to form the prediction set. We applied the proposed algorithm to several standard medical imaging classification datasets. We show that our method outperforms current methods by a large margin, in terms of the average size of the prediction set, while maintaining the required coverage.

Cited by

• Efficient Conformal Prediction for Regression Models under Label Noise

Related

• Conformal Prediction of Classifiers with Many Classes based on Noisy Labels
• Efficient Conformal Prediction for Regression Models under Label Noise
• A conformalized learning of a prediction set with applications to medical imaging classification
• RR-CP: Reliable-Region-Based Conformal Prediction for Trustworthy Medical Image Classification
• A Critical Perspective on Finite Sample Conformal Prediction Theory in Medical Applications