Mirror averaging with sparsity priors

2010/03/31 by Arnak S. Dalalyan, Alexandre B. Tsybakov · 3 citations
Mathematics · #math.ST #stat.TH

paper · pdf · doi:10.3150/11-bej361

published as Bernoulli 18, 3 (2012) 914-944 · Published in at http://dx.doi.org/10.3150/11-BEJ361 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

arxiv created 2012/08/17 · arxiv updated 2013/03/25

Abstract

We consider the problem of aggregating the elements of a possibly infinite dictionary for building a decision procedure that aims at minimizing a given criterion. Along with the dictionary, an independent identically distributed training sample is available, on which the performance of a given procedure can be tested. In a fairly general set-up, we establish an oracle inequality for the Mirror Averaging aggregate with any prior distribution. By choosing an appropriate prior, we apply this oracle inequality in the context of prediction under sparsity assumption for the problems of regression with random design, density estimation and binary classification.

Cited by

• Confidence Sequences for Generalized Linear Models via Regret Analysis
• Kullback-Leibler excess risk bounds for exponential weighted aggregation in Generalized linear models
• Optimal sparse phase retrieval via a quasi-Bayesian approach