Unbiased Measurement of Feature Importance in Tree-Based Methods

2019/03/12 by Zhou, Zhengze, Hooker, Giles · 1 citation
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1903.05179

Abstract

We propose a modification that corrects for split-improvement variable importance measures in Random Forests and other tree-based methods. These methods have been shown to be biased towards increasing the importance of features with more potential splits. We show that by appropriately incorporating split-improvement as measured on out of sample data, this bias can be corrected yielding better summaries and screening tools.

Cited by

• Provable Recovery of Locally Important Signed Features and Interactions from Random Forest

Related

• Feature Importance Measurement based on Decision Tree Sampling
• From unbiased MDI Feature Importance to Explainable AI for Trees
• A Notion of Feature Importance by Decorrelation and Detection of Trends by Random Forest Regression
• Bias in Random Forest Variable Importance Measures: Illustrations, Sources and a Solution
• Single Sample Feature Importance: An Interpretable Algorithm for Low-Level Feature Analysis