Teaching Uncertainty Quantification in Machine Learning through Use Cases

2021/08/19 by Matías Valdenegro-Toro, Valdenegro-Toro, Matias
Computer Science · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2108.08712

openalex publication_date 2021/08/19 · openalex created_date 2021/08/30 · openalex updated_date 2026/07/28

Abstract

Uncertainty in machine learning is not generally taught as general knowledge in Machine Learning course curricula. In this paper we propose a short curriculum for a course about uncertainty in machine learning, and complement the course with a selection of use cases, aimed to trigger discussion and let students play with the concepts of uncertainty in a programming setting. Our use cases cover the concept of output uncertainty, Bayesian neural networks and weight distributions, sources of uncertainty, and out of distribution detection. We expect that this curriculum and set of use cases motivates the community to adopt these important concepts into courses for safety in AI.

Citations

• A Conceptual Introduction to Hamiltonian Monte Carlo
• On Calibration of Modern Neural Networks
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Related

• Uncertainty in Machine Learning
• A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning
• Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification
• Sources of Uncertainty in Supervised Machine Learning -- A Statisticians' View
• On the Need to Align Intent and Implementation in Uncertainty Quantification for Machine Learning