Narrowing the Gap: Random Forests In Theory and In Practice

2013/10/04 by Misha Denil, Denil, Misha, David S. Matheson +3 · 5 citations
Computer Science · #Data Management and Algorithms #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1310.1415

openalex publication_date 2013/10/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Despite widespread interest and practical use, the theoretical properties of random forests are still not well understood. In this paper we contribute to this understanding in two ways. We present a new theoretically tractable variant of random regression forests and prove that our algorithm is consistent. We also provide an empirical evaluation, comparing our algorithm and other theoretically tractable random forest models to the random forest algorithm used in practice. Our experiments provide insight into the relative importance of different simplifications that theoreticians have made to obtain tractable models for analysis.

Citations

• Consistency of Online Random Forests
• Analysis of a Random Forests Model

Cited by

• Multivariate Uncertainty Quantification with Tomographic Quantile Forests
• Differentially- and non-differentially-private random decision trees
• Estimation and Inference with Trees and Forests in High Dimensions
• Targeted tuning of random forests for quantile estimation and prediction intervals
• On the Optimality of Trees Generated by ID3

Related

• Analysis of a Random Forests Model
• Grafting: Making Random Forests Consistent
• Random Forests: some methodological insights
• Random forests and kernel methods
• A Random Forest Guided Tour