Optimization of Oblivious Decision Tree Ensembles Evaluation for CPU

2022/11/01 by Alexey Mironov, Mironov, Alexey, I. M. Khuziev +1
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2211.00391

openalex publication_date 2022/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

CatBoost is a popular machine learning library. CatBoost models are based on oblivious decision trees, making training and evaluation rapid. CatBoost has many applications, and some require low latency and high throughput evaluation. This paper investigates the possibilities for improving CatBoost's performance in single-core CPU computations. We explore the new features provided by the AVX instruction sets to optimize evaluation. We increase performance by 20-40% using AVX2 instructions without quality impact. We also introduce a new trade-off between speed and quality. Using float16 for leaf values and AVX-512 instructions, we achieve 50-70% speed-up.

Related

• Optimization of Decision Tree Evaluation Using SIMD Instructions
• Register Your Forests: Decision Tree Ensemble Optimization by Explicit CPU Register Allocation
• TREE: Tree Regularization for Efficient Execution
• Fast Inference of Tree Ensembles on ARM Devices
• Runtime Optimizations for Prediction with Tree-Based Models