TableDreamer: Progressive and Weakness-guided Data Synthesis from Scratch for Table Instruction Tuning

2025/06/10 by Mingyu Zheng, Zhifan Feng, Zheng, Mingyu +11 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Machine Learning (cs.LG) #Natural Language Processing Techniques #Parallel Computing and Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2506.08646

openalex publication_date 2025/06/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Despite the commendable progress of recent LLM-based data synthesis methods, they face two limitations in generating table instruction tuning data. First, they can not thoroughly explore the vast input space of table understanding tasks, leading to limited data diversity. Second, they ignore the weaknesses in table understanding ability of the target LLM and blindly pursue the increase of data quantity, resulting in suboptimal data efficiency. In this paper, we introduce a progressive and weakness-guided data synthesis framework tailored for table instruction tuning, named TableDreamer, to mitigate the above issues. Specifically, we first synthesize diverse tables and related instructions as seed data, and then perform an iterative exploration of the input space under the guidance of the newly identified weakness data, which eventually serve as the final training data for fine-tuning the target LLM. Extensive experiments on 10 tabular benchmarks demonstrate the effectiveness of the proposed framework, which boosts the average accuracy of Llama3.1-8B-instruct by 11.62% (49.07% to 60.69%) with 27K GPT-4o synthetic data and outperforms state-of-the-art data synthesis baselines which use more training data. The code and data is available at https://github.com/SpursGoZmy/TableDreamer

Citations

• Rethinking Table Instruction Tuning
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Synth-Align: Improving Trustworthiness in Vision-Language Model with Synthetic Preference Data Alignment
• LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
• A Survey on LLM-as-a-Judge
• TableGPT2: A Large Multimodal Model with Tabular Data Integration
• MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct
• Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution
• TableBench: A Comprehensive and Complex Benchmark for Table Question Answering
• On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey
• GenQA: Generating Millions of Instructions from a Handful of Prompts
• Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
• HelpSteer2: Open-source dataset for training top-performing reward models
• TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy
• FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering
• TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios
• Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
• Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs
• Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMs
• Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding
• Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
• TableLlama: Towards Open Large Generalist Models for Tables
• Tabular Representation, Noisy Operators, and Impacts on Table Structure Understanding Tasks in LLMs
• Table-GPT: Table-tuned GPT for Diverse Table Tasks
• An Empirical Study of Instruction-tuning Large Language Models in Chinese
• Code Llama: Open Foundation Models for Code
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation
• LIMA: Less Is More for Alignment
• Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Framework
• Self-Instruct: Aligning Language Models with Self-Generated Instructions
• ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples
• Large Language Models are few(1)-shot Table Reasoners
• Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
• Knowledge Distillation from A Stronger Teacher
• AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industry
• FeTaQA: Free-form Table Question Answering
• Measuring Massive Multitask Language Understanding
• INFOTABS: Inference on Tables as Semi-structured Data
• TabFact: A Large-scale Dataset for Table-based Fact Verification
• TabMCQ: A Dataset of General Knowledge Tables and Multiple-choice Questions
• Compositional Semantic Parsing on Semi-Structured Tables

Cited by

• Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluation

Related

• Instruction Tuning of Large Language Models for Tabular Data Generation-in One Day
• Rethinking Table Instruction Tuning
• Enhancing Table Representations with LLM-powered Synthetic Data Generation
• Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search
• What Really Matters for Table LLMs? A Meta-Evaluation of Model and Data Effects