Rethinking Uncertainty Quantification and Entanglement in Image Segmentation

2026/03/31 by Jakob Lønborg Christensen, Vedrana Andersen Dahl, Morten Rieger Hannemose +2
Computer Science · #cs.CV

paper · pdf · doi:10.48550/arxiv.2603.18792

arxiv created 2026/08/04 · arxiv updated 2026/08/05

Abstract

Uncertainty quantification (UQ) is crucial in safety-critical applications such as medical image segmentation. Total uncertainty is typically decomposed into data-related aleatoric uncertainty (AU) and model-related epistemic uncertainty (EU). Many methods exist for modeling AU (such as Probabilistic UNet, Diffusion) and EU (such as ensembles, MC Dropout), but it is unclear how they interact when combined. Additionally, recent work has revealed substantial entanglement between AU and EU, undermining the interpretability and practical usefulness of the decomposition. We present a comprehensive empirical study covering a broad range of AU-EU model combinations, propose a metric to quantify uncertainty entanglement, and evaluate both across downstream UQ tasks. Ensembles consistently exhibit lower entanglement and superior performance. Softmax models usually beat other AU methods, except in calibration where the results are dataset-dependent. A softmax ensemble performs remarkably well on all tasks. Finally, we analyze potential sources of uncertainty entanglement and outline directions for mitigating this effect.

Citations

• HybridFlow: Quantification of Aleatoric and Epistemic Uncertainty with a Single Hybrid Model
• Diffusion Based Ambiguous Image Segmentation
• A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation
• Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion
• (Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Models
• The Epistemic Uncertainty Hole: an issue of Bayesian Neural Networks
• Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks
• Estimating Epistemic and Aleatoric Uncertainty with a Single Model
• ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation
• Introducing an Improved Information-Theoretic Measure of Predictive Uncertainty
• Ambiguous Medical Image Segmentation using Diffusion Models
• Laplacian Segmentation Networks Improve Epistemic Uncertainty Quantification
• On the Importance of Noise Scheduling for Diffusion Models
• Simple diffusion: End-to-end diffusion for high resolution images
• Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?
• Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation
• Denoising Diffusion Implicit Models
• Denoising Diffusion Probabilistic Models
• Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks
• A Probabilistic U-Net for Segmentation of Ambiguous Images
• Averaging Weights Leads to Wider Optima and Better Generalization
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Playing for Data: Ground Truth from Computer Games
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Efficient Object Localization Using Convolutional Networks
• The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans