2023/10/04 by Jiří Navrátil, Benjamin Elder, Navratil, Jiri +7
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Fault Detection and Control Systems #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2310.03158
openalex publication_date 2023/10/04 · openalex created_date 2023/10/08 · openalex updated_date 2026/07/28
Accurate quantification of model uncertainty has long been recognized as a fundamental requirement for trusted AI. In regression tasks, uncertainty is typically quantified using prediction intervals calibrated to an ad-hoc operating point, making evaluation and comparison across different studies relatively difficult. Our work leverages: (1) the concept of operating characteristics curves and (2) the notion of a gain over a null reference, to derive a novel operating point agnostic assessment methodology for prediction intervals. The paper defines the Uncertainty Characteristics Curve and demonstrates its utility in selected scenarios. We argue that the proposed method addresses the current need for comprehensive assessment of prediction intervals and thus represents a valuable addition to the uncertainty quantification toolbox.