2017/01/09 by Torsten Hothorn, Achim Zeileis, Hothorn, Torsten +1 · 10 citations
Computer Science · Mathematics · #Artificial intelligence #Computer science #Conditional expectation #Conditional probability distribution #Data mining #Data transformation #Econometrics #Estimator #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Inference #Machine Learning (stat.ML) #Machine Learning and Data Classification #Machine learning #Mathematics #Methodology (stat.ME) #Statistical Methods and Inference #Statistics #Transformation (genetics) #stat.ME #stat.ML
paper · pdf · doi:10.48550/arxiv.1701.02110
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2017/01/09 · arxiv created 2018/01/08 · arxiv updated 2018/01/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Regression models for supervised learning problems with a continuous target are commonly understood as models for the conditional mean of the target given predictors. This notion is simple and therefore appealing for interpretation and visualisation. Information about the whole underlying conditional distribution is, however, not available from these models. A more general understanding of regression models as models for conditional distributions allows much broader inference from such models, for example the computation of prediction intervals. Several random forest-type algorithms aim at estimating conditional distributions, most prominently quantile regression forests (Meinshausen, 2006, JMLR). We propose a novel approach based on a parametric family of distributions characterised by their transformation function. A dedicated novel "transformation tree" algorithm able to detect distributional changes is developed. Based on these transformation trees, we introduce "transformation forests" as an adaptive local likelihood estimator of conditional distribution functions. The resulting models are fully parametric yet very general and allow broad inference procedures, such as the model-based bootstrap, to be applied in a straightforward way.