Uncertainty in Physics and AI: Taxonomy, Quantification, and Validation

2026/05/31 by Manuel Haußmann, Ramon Winterhalder, Maria Ubiali
Mathematics · Physics and Astronomy · #stat.ML #astro-ph.CO #astro-ph.GA #hep-ex #hep-ph

paper · pdf

arxiv created 2026/07/29 · arxiv updated 2026/07/31

Abstract

Reliable uncertainty quantification is essential for the use of machine learning in physics, where scientific discoveries depend on validated probabilistic statements. We provide a structured overview of uncertainty quantification in ML for physics, introducing a unified taxonomy of uncertainty and clarifying the interpretation of predictive and inference uncertainties across frequentist and Bayesian frameworks. We discuss principled validation tools, including coverage, calibration, bias tests, and proper scoring rules, and illustrate them with simple regression and classification examples.

Citations

• An Introduction to Bayesian and Frequentist Simulation-Based Inference with Machine Learning
• Another Fit Bites the Dust: Conformal Prediction as a Calibration Standard for Machine Learning in High-Energy Physics
• Amplitude Surrogates for Multi-Jet Processes
• Gaussian Processes for Inferring Parton Distributions
• Colibri: A new tool for fast-flying PDF fits
• Towards a foundation model for astrophysical source detection: An End-to-End Gamma-Ray Data Analysis Pipeline Using Deep Learning
• Amplitude Uncertainties Everywhere All at Once
• Angular Coefficients from Interpretable Machine Learning with Symbolic Regression
• A linear PDF model for Bayesian inference
• Why Machine Learning Models Fail to Fully Capture Epistemic Uncertainty
• Unbinned inclusive cross-section measurements with machine-learned systematic uncertainties
• Evaluating the faithfulness of PDF uncertainties in the presence of inconsistent data
• Bayesian Computation in Deep Learning
• Evidential Deep Learning for Uncertainty Quantification and Out-of-Distribution Detection in Jet Identification using Deep Neural Networks
• Rethinking Aleatoric and Epistemic Uncertainty
• Anomalous electroweak physics unraveled via evidential deep learning
• Accurate Surrogate Amplitudes with Calibrated Uncertainties
• Precision calibration of calorimeter signals in the ATLAS experiment using an uncertainty-aware neural network
• Deep learning-driven likelihood-free parameter inference for 21-cm forest observations
• A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning
• A critical study of the Monte Carlo replica method
• Bayesian Inference with Gaussian Processes for the Determination of Parton Distribution Functions
• PINNferring the Hubble Function with Uncertainties
• Calibrating gravitational-wave search algorithms with conformal prediction
• BlackJAX: Composable Bayesian inference in JAX
• Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI
• Tractable Function-Space Variational Inference in Bayesian Neural Networks
• ELUQuant: Event-Level Uncertainty Quantification in Deep Inelastic Scattering
• Sources of Uncertainty in Supervised Machine Learning -- A Statisticians' View
• Loop Amplitudes from Precision Networks
• Equinox: neural networks in JAX via callable PyTrees and filtered transformations
• User-friendly Introduction to PAC-Bayes Bounds
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• An open-source machine learning framework for global analyses of parton distributions
• A Survey of Uncertainty in Deep Neural Networks
• Laplace Redux -- Effortless Bayesian Deep Learning
• Repulsive Deep Ensembles are Bayesian
• Multivariate Deep Evidential Regression
• A review of uncertainty quantification in deep learning: Techniques, applications and challenges
• Functional Variational Bayesian Neural Networks
• Evidential Deep Learning to Quantify Classification Uncertainty
• Predictive Uncertainty Estimation via Prior Networks
• Modeling Smooth Backgrounds and Generic Localized Signals with Gaussian Processes
• On Calibration of Modern Neural Networks
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• A Conceptual Introduction to Hamiltonian Monte Carlo
• Bayesian Methods in Cosmology
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Variational Inference: A Review for Statisticians
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Stochastic Gradient Hamiltonian Monte Carlo
• Bayesian Active Learning for Classification and Preference Learning
• Data analysis recipes: Fitting a model to data
• Strictly Proper Scoring Rules, Prediction, and Estimation
• VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY

Related

• A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning
• On the Need to Align Intent and Implementation in Uncertainty Quantification for Machine Learning
• Uncertainty Quantification for Data-Driven Machine Learning Models in Nuclear Engineering Applications: Where We Are and What Do We Need?
• Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification
• Sources of Uncertainty in Supervised Machine Learning -- A Statisticians' View