2020/01/26 by Yingshi Chen, Chen, Yingshi · 2 citations
Computer Science · Mathematics · #Artificial intelligence #Boosting (machine learning) #Combinatorics #Computer science #Data Stream Mining Techniques #FOS: Computer and information sciences #Gradient boosting #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Mathematics #Neural Networks and Applications #Tree (set theory) #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2001.09419
published in arXiv (Cornell University) (Cornell University) · 6 Pages,1 Figure
arxiv created 2020/01/26 · openalex publication_date 2020/01/26 · arxiv updated 2020/01/28 · openalex created_date 2020/01/30 · openalex updated_date 2026/07/28
Gradient boosted decision trees (GBDT) is the leading algorithm for many commercial and academic data applications. We give a deep analysis of this algorithm, especially the histogram technique, which is a basis for the regulized distribution with compact support. We present three new modifications. 1) Share memory technique to reduce memory usage. In many cases, it only need the data source itself and no extra memory. 2) Implicit merging for "merge overflow problem"."merge overflow" means that merge some small datasets to huge datasets, which are too huge to be solved. By implicit merging, we just need the original small datasets to train the GBDT model. 3) Adaptive resize algorithm of histogram bins to improve accuracy. Experiments on two large Kaggle competitions verified our methods. They use much less memory than LightGBM and have higher accuracy. We have implemented these algorithms in an open-source package LiteMORT. The source codes are available at https://github.com/closest-git/LiteMORT