2019/04/02 by Konstantin Posch, Posch, Konstantin, Jürgen Pilz +1 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification
paper · pdf · doi:10.48550/arxiv.1904.01334
openalex publication_date 2019/04/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this article a novel approach for training deep neural networks using\nBayesian techniques is presented. The Bayesian methodology allows for an easy\nevaluation of model uncertainty and additionally is robust to overfitting.\nThese are commonly the two main problems classical, i.e. non-Bayesian,\narchitectures have to struggle with. The proposed approach applies variational\ninference in order to approximate the intractable posterior distribution. In\nparticular, the variational distribution is defined as product of multiple\nmultivariate normal distributions with tridiagonal covariance matrices. Each\nsingle normal distribution belongs either to the weights, or to the biases\ncorresponding to one network layer. The layer-wise a posteriori variances are\ndefined based on the corresponding expectation values and further the\ncorrelations are assumed to be identical. Therefore, only a few additional\nparameters need to be optimized compared to non-Bayesian settings. The novel\napproach is successfully evaluated on basis of the popular benchmark datasets\nMNIST and CIFAR-10.\n