Improved dimension dependence in the Bernstein von Mises Theorem via a new Laplace approximation bound

2023/08/14 by Anya Katsevich, Katsevich, Anya · 5 citations
Mathematics · #FOS: Mathematics #Markov Chains and Monte Carlo Methods #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2308.06899

openalex publication_date 2023/08/14 · openalex created_date 2023/08/16 · openalex updated_date 2026/07/28

Abstract

The Bernstein-von Mises theorem (BvM) gives conditions under which the posterior distribution of a parameter θ∈Θ⊆\mathbb R^d based on n independent samples is asymptotically normal. In the high-dimensional regime, a key question is to determine the growth rate of d with n required for the BvM to hold. We show that up to a model-dependent coefficient, n≫ d² suffices for the BvM to hold in two settings: arbitrary generalized linear models, which include exponential families as a special case, and multinomial data, in which the parameter of interest is an unknown probability mass functions on d+1 states. Our results improve on the tightest previously known condition for posterior asymptotic normality, n≫ d³. Our statements of the BvM are nonasymptotic, taking the form of explicit high-probability bounds. To prove the BvM, we derive a new simple and explicit bound on the total variation distance between a measure π∝ e^-nf on Θ⊆\mathbb R^d and its Laplace approximation.

Cited by

• Saddle Point Approximation and Central Limit Theorem for Densities in high dimensions
• Weak convergence of Bayes estimators under general loss functions
• Discrete to continuum limits in Bayesian inverse problems
• A unified theory of the high-dimensional Laplace approximation with application to Bayesian inverse problems
• Canonical Bayesian Linear System Identification

Related

• Bernstein - von Mises Theorem for growing parameter dimension
• Normal approximation for the posterior in exponential families
• A deterministic and computable Bernstein-von Mises theorem
• Early Highlights in the History of the Bernstein-von Mises Theorem
• Accuracy of Gaussian approximation in nonparametric Bernstein -- von Mises Theorem