A Note on the PAC Bayesian Theorem

2004/11/30 by Andreas Maurer, Maurer, Andreas · 19 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Gaussian Processes and Bayesian Inference #Machine Learning and Algorithms #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.cs/0411099

9 pages

arxiv created 2004/11/30 · arxiv updated 2009/12/01

Abstract

We prove general exponential moment inequalities for averages of [0,1]-valued iid random variables and use them to tighten the PAC Bayesian Theorem. The logarithmic dependence on the sample count in the enumerator of the PAC Bayesian bound is halved.

Citations

• A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations

Cited by

• Are Flat Minima an Illusion?
• PAC-Bayes Analysis Beyond the Usual Bounds
• PAC-Bayes with Backprop
• A Framework for Bounding Deterministic Risk with PAC-Bayes: Applications to Majority Votes
• PAC-Bayes-Chernoff bounds for unbounded losses
• Split-kl and PAC-Bayes-split-kl Inequalities for Ternary Random Variables
• A New PAC-Bayesian Perspective on Domain Adaptation
• Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Vote
• Symmetries in PAC-Bayesian Learning
• PAC-Bayesian Bounds on Constrained f-Entropic Risk Measures
• Some theoretical improvements on the tightness of PAC-Bayes risk certificates for neural networks
• Generalization of Gibbs and Langevin Monte Carlo Algorithms in the Interpolation Regime
• Non-Vacuous Generalization Bounds: Can Rescaling Invariances Help?
• Generalization Guarantees for Imitation Learning
• Unifying Variational Inference and PAC-Bayes for Supervised Learning that Scales
• Learning via Wasserstein-Based High Probability Generalisation Bounds
• Efron-Stein PAC-Bayesian Inequalities
• Latent Simplex Position Model: High Dimensional Multi-view Clustering with Uncertainty Quantification
• PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes

Related

• PAC-Bayesian Inequalities for Martingales
• Dimension-free PAC-Bayesian bounds for the estimation of the mean of a random vector
• A unified recipe for deriving (time-uniform) PAC-Bayes bounds
• A note on generalization bounds for losses with finite moments
• PAC-Bayes Mini-tutorial: A Continuous Union Bound