Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity

2008/03/31 by Arnak Dalalyan, Alexandre Tsybakov · 3 citations
Mathematics · #math.ST #stat.TH

paper · pdf · doi:10.1007/s10994-008-5051-0

published as Machine Learning 72, 1-2 (2008) 39-61

arxiv created 2013/03/22 · arxiv updated 2013/03/25

Abstract

We study the problem of aggregation under the squared loss in the model of regression with deterministic design. We obtain sharp PAC-Bayesian risk bounds for aggregates defined via exponential weights, under general assumptions on the distribution of errors and on the functions to aggregate. We then apply these results to derive sparsity oracle inequalities.

Cited by

• Confidence Sequences for Generalized Linear Models via Regret Analysis
• Kullback-Leibler excess risk bounds for exponential weighted aggregation in Generalized linear models
• Optimal sparse phase retrieval via a quasi-Bayesian approach