A Technical Exploration of Causal Inference with Hybrid LLM Synthetic Data

2025/10/31 by Yichen Xu, Kim, Dana, Xu, Yichen +2 · 1 citation
Mathematics · Computer Science · #Advanced Causal Inference Techniques #Explainable Artificial Intelligence (XAI) #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2511.00318

Abstract

Large Language Models (LLMs) offer a flexible means to generate synthetic tabular data, yet existing approaches often fail to preserve key causal parameters such as the average treatment effect (ATE). In this technical exploration, we first demonstrate that state-of-the-art synthetic data generators, both GAN- and LLM-based, can achieve high predictive fidelity while substantially misestimating causal effects. To address this gap, we propose a hybrid generation framework that combines model-based covariate synthesis (monitored via distance-to-closest-record filtering) with separately learned propensity and outcome models, thereby ensuring that (W, A, Y) triplets retain their underlying causal structure. We further introduce a synthetic pairing strategy to mitigate positivity violations and a realistic evaluation protocol that leverages unlimited synthetic samples to benchmark traditional estimators (IPTW, AIPW, substitution) under complex covariate distributions. This work lays the groundwork for LLM-powered data pipelines that support robust causal analysis. Our code is available at https://github.com/Xyc-arch/llm-synthetic-for-causal-inference.git.

Citations

• Efficient Randomized Experiments Using Foundation Models
• Synthetic Oversampling: Theory and A Practical Approach Using LLMs to Address Data Imbalance
• Large Language Models and Causal Inference in Collaboration: A Survey
• Language Models are Realistic Tabular Data Generators
• Modeling Tabular data using Conditional GAN

Cited by

• Harnessing Synthetic Data from Generative AI for Statistical Inference

Related

• Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities
• End-To-End Causal Effect Estimation from Unstructured Natural Language Data
• Generating Synthetic Text Data to Evaluate Causal Inference Methods
• ALCM: Autonomous LLM-Augmented Causal Discovery Framework
• Causal Reasoning and Large Language Models: Opening a New Frontier for Causality