Confidence Intervals for the Generalisation Error of Random Forests

2022/01/26 by Samyak Rajanala, Rajanala, Samyak, Stephen Bates +5
Computer Science · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning and Data Classification #Methodology (stat.ME) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2201.11210

openalex publication_date 2022/01/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Out-of-bag error is commonly used as an estimate of generalisation error in ensemble-based learning models such as random forests. We present confidence intervals for this quantity using the delta-method-after-bootstrap and the jackknife-after-bootstrap techniques. These methods do not require growing any additional trees. We show that these new confidence intervals have improved coverage properties over the naive confidence interval, in real and simulated examples.

Related

• Confidence intervals for the random forest generalization error
• Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife
• Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests
• Stability of Random Forests and Coverage of Random-Forest Prediction Intervals
• Interpretable Machines: Constructing Valid Prediction Intervals with Random Forests