2018/05/12 by Diego Vergara, Sergio Hernández, Vergara, Diego +6
Computer Science · Decision Sciences · Mathematics · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Probabilistic and Robust Engineering Design #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1805.04756
26 Pages, 12 Figures, version 3, to appear in Soft Computing, Author preprint
openalex publication_date 2018/05/12 · arxiv created 2019/07/02 · arxiv updated 2019/07/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Estimating predictive uncertainty is crucial for many computer vision tasks, from image classification to autonomous driving systems. Hamiltonian Monte Carlo (HMC) is an sampling method for performing Bayesian inference. On the other hand, Dropout regularization has been proposed as an approximate model averaging technique that tends to improve generalization in large scale models such as deep neural networks. Although, HMC provides convergence guarantees for most standard Bayesian models, it does not handle discrete parameters arising from Dropout regularization. In this paper, we present a robust methodology for improving predictive uncertainty in classification problems, based on Dropout and Hamiltonian Monte Carlo. Even though Dropout induces a non-smooth energy function with no such convergence guarantees, the resulting discretization of the Hamiltonian proves empirical success. The proposed method allows to effectively estimate the predictive accuracy and to provide better generalization for difficult test examples.