Towards Better Serialization of Tabular Data for Few-shot Classification with Large Language Models

2023/12/18 by Sukriti Jaitly, Tanay Shah, Jaitly, Sukriti +5 · 2 citations
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Data Quality and Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2312.12464

openalex publication_date 2023/12/18 · openalex created_date 2023/12/23 · openalex updated_date 2026/07/28

Abstract

We present a study on the integration of Large Language Models (LLMs) in tabular data classification, emphasizing an efficient framework. Building upon existing work done in TabLLM (arXiv:2210.10723), we introduce three novel serialization techniques, including the standout LaTeX serialization method. This method significantly boosts the performance of LLMs in processing domain-specific datasets, Our method stands out for its memory efficiency and ability to fully utilize complex data structures. Through extensive experimentation, including various serialization approaches like feature combination and importance, we demonstrate our work's superiority in accuracy and efficiency over traditional models.

Cited by

• Serialized EHR make for good text representations
• Large Language Models as Universal Predictors? An Empirical Study on Small Tabular Datasets

Related

• TabLLM: Few-shot Classification of Tabular Data with Large Language Models
• Text Serialization and Their Relationship with the Conventional Paradigms of Tabular Machine Learning
• TabuLa: Harnessing Language Models for Tabular Data Synthesis
• Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challenges
• Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution