Adversarial random forests for density estimation and generative modeling

2022/05/19 by David S. Watson, Kristin Blesch, Jan Kapar +1 · 1 voice · 2 citations
Mathematics · Computer Science · #stat.ML #cs.AI #cs.LG #stat.CO

paper · pdf

arxiv published 2022/05/19 · arxiv updated 2023/03/13

Abstract

We propose methods for density estimation and data synthesis using a novel form of unsupervised random forests. Inspired by generative adversarial networks, we implement a recursive procedure in which trees gradually learn structural properties of the data through alternating rounds of generation and discrimination. The method is provably consistent under minimal assumptions. Unlike classic tree-based alternatives, our approach provides smooth (un)conditional densities and allows for fully synthetic data generation. We achieve comparable or superior performance to state-of-the-art probabilistic circuits and deep learning models on various tabular data benchmarks while executing about two orders of magnitude faster on average. An accompanying R package, arf, is available on CRAN.

Cited by

• Machine learning models for estimating counterfactuals in a single-arm inflammatory bowel disease study
• Vine Copulas for Analyzing Multivariate Conditional Dependencies in Electronic Health Records Data

Discussions

• Side note: I attended a seminar this week about a new method called Adversarial Random Forest, which made me excited. The group that develops the method cares about statistical consistency and they ha [bsky, 0 points, 0 comments]

Related

• arfpy: A python package for density estimation and generative modeling with adversarial random forests
• Generative Forests
• LED: Latent Variable-based Estimation of Density
• Exploiting random projections and sparsity with random forests and gradient boosting methods -- Application to multi-label and multi-output learning, random forest model compression and leveraging input sparsity
• Best-scored Random Forest Density Estimation