Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees

2023/09/18 by Alexia Jolicoeur-Martineau, Jolicoeur-Martineau, Alexia, Kilian Fatras +3 · 2 voices · 4 citations
#cs.LG

paper · pdf · doi:10.48550/arxiv.2309.09968

Abstract

Tabular data is hard to acquire and is subject to missing values. This paper introduces a novel approach for generating and imputing mixed-type (continuous and categorical) tabular data utilizing score-based diffusion and conditional flow matching. In contrast to prior methods that rely on neural networks to learn the score function or the vector field, we adopt XGBoost, a widely used Gradient-Boosted Tree (GBT) technique. To test our method, we build one of the most extensive benchmarks for tabular data generation and imputation, containing 27 diverse datasets and 9 metrics. Through empirical evaluation across the benchmark, we demonstrate that our approach outperforms deep-learning generation methods in data generation tasks and remains competitive in data imputation. Notably, it can be trained in parallel using CPUs without requiring a GPU. Our Python and R code is available at https://github.com/SamsungSAILMontreal/ForestDiffusion.

Cited by

• Flow Matching for Tabular Data Synthesis
• Boosting Predictive Performance on Tabular Data through Data Augmentation with Latent-Space Flow-Based Diffusion
• An Interdisciplinary and Cross-Task Review on Missing Data Imputation
• TabImpute: Universal Zero-Shot Imputation for Tabular Data
• Privacy-Preserving Tabular Synthetic Data Generation Using TabularARGN

Discussions

• Diffusion and flow matching generative models implemented with XGBoost instead of neural networks! arxiv.org/abs/2309.09968 [bsky, 15 points, 0 comments]
• Related to this. There's a cool paper exploiting the fact that trees can handle missingness well to impute data using unconditional diffusions. Definitely worth taking a look! arxiv.org/abs/2309.0996 [bsky, 3 points, 2 comments]

Related

• Unmasking Trees for Tabular Data
• BUFF: Boosted Decision Tree based Ultra-Fast Flow matching
• Tabular Data Generation Models: An In-Depth Survey and Performance Benchmarks with Extensive Tuning
• Scaling Up Diffusion and Flow-based XGBoost Models
• Revisiting Deep Learning Models for Tabular Data