2022/06/02 by Brian Staber, Staber, Brian, Sébastien da Veiga +1
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #Classical Physics (physics.class-ph) #FOS: Computer and information sciences #FOS: Physical sciences #Fault Detection and Control Systems #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine Learning and Data Classification #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.2206.06779
openalex publication_date 2022/06/02 · openalex created_date 2022/06/13 · openalex updated_date 2026/07/28
Due to the growing adoption of deep neural networks in many fields of science\nand engineering, modeling and estimating their uncertainties has become of\nprimary importance. Despite the growing literature about uncertainty\nquantification in deep learning, the quality of the uncertainty estimates\nremains an open question. In this work, we assess for the first time the\nperformance of several approximation methods for Bayesian neural networks on\nregression tasks by evaluating the quality of the confidence regions with\nseveral coverage metrics. The selected algorithms are also compared in terms of\npredictivity, kernelized Stein discrepancy and maximum mean discrepancy with\nrespect to a reference posterior in both weight and function space. Our\nfindings show that (i) some algorithms have excellent predictive performance\nbut tend to largely over or underestimate uncertainties (ii) it is possible to\nachieve good accuracy and a given target coverage with finely tuned\nhyperparameters and (iii) the promising kernel Stein discrepancy cannot be\nexclusively relied on to assess the posterior approximation. As a by-product of\nthis benchmark, we also compute and visualize the similarity of all algorithms\nand corresponding hyperparameters: interestingly we identify a few clusters of\nalgorithms with similar behavior in weight space, giving new insights on how\nthey explore the posterior distribution.\n