Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inference

2023/05/24 by Insung Kong, Dong‐Yoon Yang, Kong, Insung +9 · 2 citations
Computer Science · Engineering · #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #Fault Detection and Control Systems #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.2305.14765

openalex publication_date 2023/05/24 · openalex created_date 2023/05/27 · openalex updated_date 2026/07/28

Abstract

Bayesian approaches for learning deep neural networks (BNN) have been received much attention and successfully applied to various applications. Particularly, BNNs have the merit of having better generalization ability as well as better uncertainty quantification. For the success of BNN, search an appropriate architecture of the neural networks is an important task, and various algorithms to find good sparse neural networks have been proposed. In this paper, we propose a new node-sparse BNN model which has good theoretical properties and is computationally feasible. We prove that the posterior concentration rate to the true model is near minimax optimal and adaptive to the smoothness of the true model. In particular the adaptiveness is the first of its kind for node-sparse BNNs. In addition, we develop a novel MCMC algorithm which makes the Bayesian inference of the node-sparse BNN model feasible in practice.

Cited by

• Knowledge Distillation of Uncertainty using Deep Latent Factor Model
• Bayesian Neural Networks for Functional ANOVA model

Related

• Masked Bayesian Neural Networks : Computation and Optimality
• Posterior concentrations of fully-connected Bayesian neural networks with general priors on the weights
• Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Details
• Adaptive Variational Bayesian Inference for Sparse Deep Neural Network
• Bayesian Neural Networks: A Min-Max Game Framework