Simulating Posterior Bayesian Neural Networks with Dependent Weights

2025/07/29 by Apollonio, Nicola, Franzina, Giovanni, Torrisi, Giovanni Luca
#60B10 #68T07 #68W20 #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Probability (math.PR)

paper · doi:10.48550/arxiv.2507.22095

Abstract

In this paper we consider posterior Bayesian fully connected and feedforward deep neural networks with dependent weights. Particularly, if the likelihood is Gaussian, we identify the distribution of the wide width limit and provide an algorithm to sample from the network. In the shallow case we explicitly compute the distribution of the conditional output, proving that it is a Gaussian mixture. All the theoretical results are numerically validated.

Citations

• Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights
• Entropic bounds for conditionally Gaussian vectors and applications to neural networks
• Student-t processes as infinite-width limits of posterior Bayesian neural networks
• Function-Space MCMC for Bayesian Wide Neural Networks
• Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selection
• The Positivity of the Neural Tangent Kernel
• Large and moderate deviations for Gaussian neural networks
• Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processes
• Quantitative CLTs in Deep Neural Networks
• Normal approximation of Random Gaussian Neural Networks
• Gaussian random field approximation via Stein's method with applications to wide random neural networks
• Non-asymptotic approximations of Gaussian neural networks via second-order Poincaré inequalities
• Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networks
• Training Integrable Parameterizations of Deep Neural Networks in the Infinite-Width Limit
• Deep Stable neural networks: large-width asymptotics and convergence rates
• Random Neural Networks in the Infinite Width Limit as Gaussian Processes
• Random neural networks in the infinite width limit as Gaussian processes
• α-Stable convergence of heavy-tailed infinitely-wide neural networks
• What Are Bayesian Neural Network Posteriors Really Like?
• Non-asymptotic approximations of neural networks by Gaussian processes
• Bayesian Neural Network Priors Revisited
• Exact posterior distributions of wide Bayesian neural networks
• Traditional and Heavy-Tailed Self Regularization in Neural Network Models
• Gaussian Process Behaviour in Wide Deep Neural Networks
• Deep Neural Networks as Gaussian Processes

Related

• Posterior Inference on Shallow Infinitely Wide Bayesian Neural Networks under Weights with Unbounded Variance
• Posterior concentrations of fully-connected Bayesian neural networks with general priors on the weights
• Exact posterior distributions of wide Bayesian neural networks
• Improved Depth Estimation of Bayesian Neural Networks
• Walking on the Fiber: A Simple Geometric Approximation for Bayesian Neural Networks