Rethinking Aleatoric and Epistemic Uncertainty

2024/12/30 by Freddie Bickford Smith, Jannik Kossen, Smith, Freddie Bickford +9 · 14 citations
Arts and Humanities · #Epistemology, Ethics, and Metaphysics #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2412.20892

openalex publication_date 2024/12/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The ideas of aleatoric and epistemic uncertainty are widely used to reason about the probabilistic predictions of machine-learning models. We identify incoherence in existing discussions of these ideas and suggest this stems from the aleatoric-epistemic view being insufficiently expressive to capture all the distinct quantities that researchers are interested in. To address this we present a decision-theoretic perspective that relates rigorous notions of uncertainty, predictive performance and statistical dispersion in data. This serves to support clearer thinking as the field moves forward. Additionally we provide insights into popular information-theoretic quantities, showing they can be poor estimators of what they are often purported to measure, while also explaining how they can still be useful in guiding data acquisition.

Cited by

• Controllable Diversity in Normalization-Based Implicit Ensembles via Softmax-Temperature Modulation
• Unreliable Uncertainty Estimates with Monte Carlo Dropout
• Beyond Accuracy: An Empirical Study of Uncertainty Estimation in Imputation
• The Illusion of Certainty: Uncertainty Quantification for LLMs Fails under Ambiguity
• Frequentist Validity of Epistemic Uncertainty Estimators
• A Geometric Approach to Optimal Experimental Design
• Automated C-Arm Positioning via Conformal Landmark Localization
• HybridFlow: Quantification of Aleatoric and Epistemic Uncertainty with a Single Hybrid Model
• A decision-theoretic framework for uncertainty quantification in epidemiological modelling
• Uncertainty in Physics and AI: Taxonomy, Quantification, and Validation
• Prediction-Oriented Subsampling from Data Streams
• The Architecture of Trust: A Framework for AI-Augmented Real Estate Valuation in the Era of Structured Data
• Flow Stochastic Segmentation Networks
• Disentangling Doubt in Deep Causal AI

Related

• Sources of Uncertainty in Supervised Machine Learning -- A Statisticians' View
• Why Machine Learning Models Fail to Fully Capture Epistemic Uncertainty
• Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?
• Epistemic Uncertainty Sampling
• Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rules