Obtaining Calibrated Probabilities from Boosting

2012/07/04 by Alexandru Niculescu-Mizil, Rich Caruana, Niculescu-Mizil, Alexandru +1 · 6 citations
Computer Science · #Explainable Artificial Intelligence (XAI) #Imbalanced Data Classification Techniques #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1207.1403

Abstract

Boosted decision trees typically yield good accuracy, precision, and ROC area. However, because the outputs from boosting are not well calibrated posterior probabilities, boosting yields poor squared error and cross-entropy. We empirically demonstrate why AdaBoost predicts distorted probabilities and examine three calibration methods for correcting this distortion: Platt Scaling, Isotonic Regression, and Logistic Correction. We also experiment with boosting using log-loss instead of the usual exponential loss. Experiments show that Logistic Correction and boosting with log-loss work well when boosting weak models such as decision stumps, but yield poor performance when boosting more complex models such as full decision trees. Platt Scaling and Isotonic Regression, however, significantly improve the probabilities predicted by

Cited by

• Hands-Free Segmentation of Medical Volumes via Binary Inputs
• Better Boosting with Bandits for Online Learning
• Misclassification cost-sensitive ensemble learning: A unifying framework
• Evidence Networks: simple losses for fast, amortized, neural Bayesian model comparison
• Non-Parametric Calibration for Classification
• Threshold Choice Methods: the Missing Link
• Solving for multi-class: a survey and synthesis

Related

• The Theory Behind Overfitting, Cross Validation, Regularization, Bagging, and Boosting: Tutorial
• Probabilistic Scores of Classifiers, Calibration is not Enough
• Calibrated Boosting-Forest
• From Point to probabilistic gradient boosting for claim frequency and severity prediction
• Boosting as a Product of Experts