TABLET: Learning From Instructions For Tabular Data

2023/04/25 by Dylan Slack, Slack, Dylan, Sameer Singh +1 · 5 citations
Computer Science · Medicine · #Adversarial Robustness in Machine Learning #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2304.13188

openalex publication_date 2023/04/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Acquiring high-quality data is often a significant challenge in training machine learning (ML) models for tabular prediction, particularly in privacy-sensitive and costly domains like medicine and finance. Providing natural language instructions to large language models (LLMs) offers an alternative solution. However, it is unclear how effectively instructions leverage the knowledge in LLMs for solving tabular prediction problems. To address this gap, we introduce TABLET, a benchmark of 20 diverse tabular datasets annotated with instructions that vary in their phrasing, granularity, and technicality. Additionally, TABLET includes the instructions' logic and structured modifications to the instructions. We find in-context instructions increase zero-shot F1 performance for Flan-T5 11b by 44% on average and 13% for ChatGPT on TABLET. Also, we explore the limitations of using LLMs for tabular prediction in our benchmark by evaluating instruction faithfulness. We find LLMs often ignore instructions and fail to predict specific instances correctly, even with examples. Our analysis on TABLET shows that, while instructions help LLM performance, learning from instructions for tabular data requires new capabilities.

Cited by

• Emergent Bias and Fairness in Multi-Agent Decision Systems
• SMITE: Enhancing Fairness in LLMs through Optimal In-Context Example Selection via Dynamic Validation
• Summarize-Exemplify-Reflect: Data-driven Insight Distillation Empowers LLMs for Few-shot Tabular Classification
• LLM Empowered Prototype Learning for Zero and Few-Shot Tasks on Tabular Data
• Tab-MIA: A Benchmark Dataset for Membership Inference Attacks on Tabular Data in LLMs

Related

• Instruction Tuning of Large Language Models for Tabular Data Generation-in One Day
• TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks
• Trompt: Towards a Better Deep Neural Network for Tabular Data
• TabuLa: Harnessing Language Models for Tabular Data Synthesis
• TabFlex: Scaling Tabular Learning to Millions with Linear Attention