A Design Space for the Critical Validation of LLM-Generated Tabular Data

2025/05/07 by Madhav Sachdeva, Christopher Narayanan, Sachdeva, Madhav +7 · 1 citation
Decision Sciences · Computer Science · #Data Quality and Management #Imbalanced Data Classification Techniques #Psychometric Methodologies and Testing

paper · pdf · doi:10.48550/arxiv.2505.04487

Abstract

LLM-generated tabular data is creating new opportunities for data-driven applications in academia, business, and society. To leverage benefits like missing value imputation, labeling, and enrichment with context-aware attributes, LLM-generated data needs a critical validation process. The number of pioneering approaches is increasing fast, opening a promising validation space that, so far, remains unstructured. We present a design space for the critical validation of LLM-generated tabular data with two dimensions: First, the Analysis Granularity dimension: from within-attribute (single-item and multi-item) to across-attribute perspectives (1 x 1, 1 x m, and n x n). Second, the Data Source dimension: differentiating between LLM-generated values, ground truth values, explanations, and their combinations. We discuss analysis tasks for each dimension cross-cut, map 19 existing validation approaches, and discuss the characteristics of two approaches in detail, demonstrating descriptive power.

Citations

• AI-Assisted Data Extraction for Systematic Reviews in Education
• A review of faithfulness metrics for hallucination assessment in Large Language Models
• Human-Centered Design Recommendations for LLM-as-a-Judge
• On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations
• Understanding Large Language Model Behaviors through Interactive Counterfactual Generation and Analysis
• Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
• Comprehensive Reassessment of Large-Scale Evaluation Outcomes in LLMs: A Multifaceted Statistical Approach
• Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey
• LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Models
• Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
• Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Can Large Language Models Be an Alternative to Human Evaluations?
• Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks
• TabLLM: Few-shot Classification of Tabular Data with Large Language Models
• Language Models are Realistic Tabular Data Generators
• Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models
• The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
• Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models

Cited by

• AwesomeLit: Towards Hypothesis Generation with Agent-Supported Literature Research

Related

• Utilizing Training Data to Improve LLM Reasoning for Tabular Understanding
• TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios
• On The Role of Prompt Construction In Enhancing Efficacy and Efficiency of LLM-Based Tabular Data Generation
• LLM-TabLogic: Preserving Inter-Column Logical Relationships in Synthetic Tabular Data via Prompt-Guided Latent Diffusion
• How well do LLMs reason over tabular data, really?