Boosting Algorithms: Regularization, Prediction and Model Fitting

2007/11/01 by Peter Bühlmann, Torsten Hothorn · 6 citations
Mathematics · #Advanced Statistical Methods and Models #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #stat.ME

paper · pdf · doi:10.1214/07-sts242

published as Statistical Science 2007, Vol. 22, No. 4, 477-505 · This paper commented in: [arXiv:0804.2757], [arXiv:0804.2770]. Rejoinder in [arXiv:0804.2777]. Published in at http://dx.doi.org/10.1214/07-STS242 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

openalex publication_date 2007/11/01 · arxiv created 2008/04/17 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28

Abstract

We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival analysis. Concepts of degrees of freedom and corresponding Akaike or Bayesian information criteria, particularly useful for regularization and variable selection in high-dimensional covariate spaces, are discussed as well. The practical aspects of boosting procedures for fitting statistical models are illustrated by means of the dedicated open-source software package mboost. This package implements functions which can be used for model fitting, prediction and variable selection. It is flexible, allowing for the implementation of new boosting algorithms optimizing user-specified loss functions.

Citations

• Comment: Boosting Algorithms: Regularization, Prediction and Model Fitting
• Comment: Boosting Algorithms: Regularization, Prediction and Model Fitting
• Rejoinder: Boosting Algorithms: Regularization, Prediction and Model Fitting
• Greedy function approximation: A gradient boosting machine.
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
• Rejoinder to "Least angle regression" by Efron et al
• Boosting With theL2Loss
• Shape Quantization and Recognition with Randomized Trees
• Unbiased Recursive Partitioning: A Conditional Inference Framework
• Matching pursuits with time-frequency dictionaries

Cited by

• Comment: Boosting Algorithms: Regularization, Prediction and Model Fitting
• Comment: Boosting Algorithms: Regularization, Prediction and Model Fitting
• Rejoinder: Boosting Algorithms: Regularization, Prediction and Model Fitting
• Predicting Recessions with Leading Indicators: Model Averaging and Selection over the Business Cycle
• Amplification of annual and diurnal cycles of alpine lightning
• Peter Bühlmann [wikipedia]