Evaluation of Representation Models for Text Classification with AutoML Tools

2021/06/24 by Brändle, Sebastian, Hanussek, Marc, Blohm, Matthias +1
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2106.12798

Abstract

Automated Machine Learning (AutoML) has gained increasing success on tabular data in recent years. However, processing unstructured data like text is a challenge and not widely supported by open-source AutoML tools. This work compares three manually created text representations and text embeddings automatically created by AutoML tools. Our benchmark includes four popular open-source AutoML tools and eight datasets for text classification purposes. The results show that straightforward text representations perform better than AutoML tools with automatically created text embeddings.

Related

• Leveraging Automated Machine Learning for Text Classification:\n Evaluation of AutoML Tools and Comparison with Human Performance
• Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark
• AutoML-guided Fusion of Entity and LLM-based Representations for Document Classification
• Benchmarking Multimodal AutoML for Tabular Data with Text Fields
• An Open Source AutoML Benchmark