2017/10/24 by Pedro G. Fonseca, Fonseca, Pedro G., Hugo Lopes +2 · 4 citations
Business, Management and Accounting · Computer Science · Economics, Econometrics and Finance · Engineering · Mathematics · #AdaBoost #Advanced Data Processing Techniques #Artificial intelligence #Benchmark (surveying) #Binary classification #Boosting (machine learning) #Brier score #Calibration #Computer science #Credit risk #Econometrics (econ.EM) #Economics #FOS: Computer and information sciences #FOS: Economics and business #Fault Detection and Control Systems #Financial Distress and Bankruptcy Prediction #Gradient boosting #Imbalanced Data Classification Techniques #Isotonic regression #Logistic regression #Machine Learning (stat.ML) #Machine learning #Mathematics #Oil and Gas Production Techniques #Probability of default #Random forest #Regression #Statistics #Support vector machine #econ.EM #stat.ML
paper · pdf · doi:10.48550/arxiv.1710.08901
published in arXiv (Cornell University) (Cornell University) · Keywords: Binary classification, Probability of Default, Calibration, Credit Risk, Isotonic Regression, Platt Scaling
arxiv created 2017/10/24 · openalex publication_date 2017/10/24 · arxiv updated 2017/10/25 · openalex created_date 2021/02/01 · openalex updated_date 2026/08/05
Binary classification is highly used in credit scoring in the estimation of probability of default. The validation of such predictive models is based both on rank ability, and also on calibration (i.e. how accurately the probabilities output by the model map to the observed probabilities). In this study we cover the current best practices regarding calibration for binary classification, and explore how different approaches yield different results on real world credit scoring data. The limitations of evaluating credit scoring models using only rank ability metrics are explored. A benchmark is run on 18 real world datasets, and results compared. The calibration techniques used are Platt Scaling and Isotonic Regression. Also, different machine learning models are used: Logistic Regression, Random Forest Classifiers, and Gradient Boosting Classifiers. Results show that when the dataset is treated as a time series, the use of re-calibration with Isotonic Regression is able to improve the long term calibration better than the alternative methods. Using re-calibration, the non-parametric models are able to outperform the Logistic Regression on Brier Score Loss.