Robustness and uncertainty: two complementary aspects of the reliability of the predictions of a classifier

2025/12/17 by Detavernier, Adrián, De Bock, Jasper
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2512.15492

Abstract

We consider two conceptually different approaches for assessing the reliability of the individual predictions of a classifier: Robustness Quantification (RQ) and Uncertainty Quantification (UQ). We compare both approaches on a number of benchmark datasets and show that there is no clear winner between the two, but that they are complementary and can be combined to obtain a hybrid approach that outperforms both RQ and UQ. As a byproduct of our approach, for each dataset, we also obtain an assessment of the relative importance of uncertainty and robustness as sources of unreliability.

Citations

• Robustness quantification: a new method for assessing the reliability of the predictions of a classifier
• Recent Advances in Adversarial Training for Adversarial Robustness
• Towards Robust Classification with Deep Generative Forests
• Measuring Robustness to Natural Distribution Shifts in Image Classification
• Aleatoric and Epistemic Uncertainty with Random Forests
• On Evaluating Adversarial Robustness

Related

• Robustness quantification: a new method for assessing the reliability of the predictions of a classifier
• The Certainty Ratio C_ρ: a novel metric for assessing the reliability of classifier predictions
• Towards an Intrinsic Definition of Robustness for a Classifier
• Towards Clear Expectations for Uncertainty Estimation
• Legitimate ground-truth-free metrics for deep uncertainty classification scoring