Squeezing Lemons with Hammers: An Evaluation of AutoML and Tabular Deep Learning for Data-Scarce Classification Applications

2024/05/13 by Ricardo Knauer, Knauer, Ricardo, Erik Rodner +1 · 1 voice
Computer Science · #Machine Learning and Data Classification #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2405.07662

openalex publication_date 2024/05/13 · openalex created_date 2024/05/15 · openalex updated_date 2026/07/28

Abstract

Many industry verticals are confronted with small-sized tabular data. In this low-data regime, it is currently unclear whether the best performance can be expected from simple baselines, or more complex machine learning approaches that leverage meta-learning and ensembling. On 44 tabular classification datasets with sample sizes ≤ 500, we find that L2-regularized logistic regression performs similar to state-of-the-art automated machine learning (AutoML) frameworks (AutoPrognosis, AutoGluon) and off-the-shelf deep neural networks (TabPFN, HyperFast) on the majority of the benchmark datasets. We therefore recommend to consider logistic regression as the first choice for data-scarce applications with tabular data and provide practitioners with best practices for further method selection.

Discussions

• "We therefore recommend to consider logistic regression as the first choice for data-scarce applications with tabular data and provide practitioners with best practices for further method selection." [bsky, 6 points, 2 comments]

Related

• PMLBmini: A Tabular Classification Benchmark Suite for Data-Scarce Applications
• Tabular Data: Is Deep Learning all you need?
• Extreme AutoML: Analysis of Classification, Regression, and NLP Performance
• TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks
• A Comprehensive Benchmark of Machine and Deep Learning Across Diverse Tabular Datasets