Uncertainty Propagation in Convolutional Neural Networks: Technical Report

2021/02/11 by Christos Tzelepis, Tzelepis, Christos, Ioannis Patras +1
Computer Science · Physics and Astronomy · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Image and Signal Denoising Methods #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2102.06064

openalex publication_date 2021/02/11 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

In this technical report we study the problem of propagation of uncertainty (in terms of variances of given uni-variate normal random variables) through typical building blocks of a Convolutional Neural Network (CNN). These include layers that perform linear operations, such as 2D convolutions, fully-connected, and average pooling layers, as well as layers that act non-linearly on their input, such as the Rectified Linear Unit (ReLU). Finally, we discuss the sigmoid function, for which we give approximations of its first- and second-order moments, as well as the binary cross-entropy loss function, for which we approximate its expected value under normal random inputs.

Related

• Uncertainty propagation in feed-forward neural network models
• Uncertainty Quantification via Stable Distribution Propagation
• On weight and variance uncertainty in neural networks for regression tasks
• Uncertainty Quantification From Scaling Laws in Deep Neural Networks
• Feed-forward Uncertainty Propagation in Belief and Neural Networks