2005/11/30 by A. Juditsky, P. Rigollet, A. B. Tsybakov · 74 citations
Decision Sciences · Mathematics · #Advanced Statistical Methods and Models #Aggregate (composite) #Construct (python library) #Estimator #Linear programming #Nonlinear system #Oracle #Risk and Portfolio Optimization #Simple (philosophy) #Statistical Methods and Inference #Type (biology) #math.ST #msc:62C20 #msc:62G05 #msc:62G08 #msc:62G20 #stat.TH
paper · pdf · doi:10.1214/07-aos546
published in The Annals of Statistics 36(5) (Institute of Mathematical Statistics) · Published in at http://dx.doi.org/10.1214/07-AOS546 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
openalex publication_date 2008/10/01 · arxiv created 2008/11/05 · arxiv updated 2009/12/01 · openalex created_date 2016/06/24 · openalex updated_date 2026/08/05
Given a finite collection of estimators or classifiers, we study the problem of model selection type aggregation, that is, we construct a new estimator or classifier, called aggregate, which is nearly as good as the best among them with respect to a given risk criterion. We define our aggregate by a simple recursive procedure which solves an auxiliary stochastic linear programming problem related to the original nonlinear one and constitutes a special case of the mirror averaging algorithm. We show that the aggregate satisfies sharp oracle inequalities under some general assumptions. The results are applied to several problems including regression, classification and density estimation.