DoubleGen: Debiased Generative Modeling of Counterfactuals

2025/09/20 by Alex Luedtke, Kenji Fukumizu, Luedtke, Alex +1 · 1 voice · 2 citations
Business, Management and Accounting · Computer Science · Physics and Astronomy · #Artificial Intelligence in Games #Business Process Modeling and Analysis #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Methodology (stat.ME) #Opinion Dynamics and Social Influence

paper · pdf · doi:10.48550/arxiv.2509.16842

openalex publication_date 2025/09/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Generative models for counterfactual outcomes face two key sources of bias. Confounding bias arises when approaches fail to account for systematic differences between those who receive the intervention and those who do not. Misspecification bias arises when methods attempt to address confounding through estimation of an auxiliary model, but specify it incorrectly. We introduce DoubleGen, a doubly robust framework that modifies generative modeling training objectives to mitigate these biases. The new objectives rely on two auxiliaries -- a propensity and outcome model -- and successfully address confounding bias even if only one of them is correct. We provide finite-sample guarantees for this robustness property. We further establish conditions under which DoubleGen achieves oracle optimality -- matching the convergence rates standard approaches would enjoy if interventional data were available -- and minimax rate optimality. We illustrate DoubleGen with three examples: diffusion models, flow matching, and autoregressive language models.

Citations

• Stochastic Gradients under Nuisances
• Statistical guarantees for denoising reflected diffusion models
• The Llama 3 Herd of Models
• Bridging Language and Items for Retrieval and Recommendation
• Combining T-learning and DR-learning: a framework for oracle-efficient estimation of causal contrasts
• Non-Vacuous Generalization Bounds for Large Language Models
• Causal normalizing flows: from theory to practice
• Error Bounds for Flow Matching Methods
• An Efficient Doubly-Robust Test for the Kernel Treatment Effect
• One-Step Estimation of Differentiable Hilbert-Valued Parameters
• Diffusion Models are Minimax Optimal Distribution Estimators
• LLaMA: Open and Efficient Foundation Language Models
• Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data
• Modeling Causal Mechanisms with Diffusion Models for Interventional and Counterfactual Queries
• Flow Matching for Generative Modeling
• Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements
• Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
• Convergence of denoising diffusion models under the manifold hypothesis
• Classifier-Free Diffusion Guidance
• Fast convergence rates for dose-response estimation
• Minimax rates for heterogeneous causal effect estimation
• Diffusion Causal Models for Counterfactual Estimation
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Conditional Image Generation with Score-Based Diffusion Models
• Palette: Image-to-Image Diffusion Models
• VACA: Design of Variational Graph Autoencoders for Interventional and Counterfactual Queries
• Automatic Debiased Machine Learning via Riesz Regression
• Semiparametric counterfactual density estimation
• Zero-Shot Text-to-Image Generation
• MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers
• Score-Based Generative Modeling through Stochastic Differential Equations
• Causal Autoregressive Flows
• A Note on High-Probability versus In-Expectation Guarantees of\n Generalization Bounds in Machine Learning
• A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation
• Deep Structural Causal Models for Tractable Counterfactual Inference
• Towards optimal doubly robust estimation of heterogeneous causal effects
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Optuna: A Next-generation Hyperparameter Optimization Framework
• MediaPipe: A Framework for Building Perception Pipelines
• The Curious Case of Neural Text Degeneration
• Improved Precision and Recall Metric for Assessing Generative Models
• Orthogonal Statistical Learning
• Language GANs Falling Short
• MMLSpark: Unifying Machine Learning Ecosystems at Massive Scales
• Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality
• Hierarchical Neural Story Generation
• Demystifying MMD GANs
• Causal Inference: A Missing Data Perspective
• Causal Inference: A Missing Data Perspective
• Quasi-Oracle Estimation of Heterogeneous Treatment Effects
• Quasi-oracle estimation of heterogeneous treatment effects
• CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training
• Attention Is All You Need
• On the multiply robust estimation of the mean of the g-functional
• Sequential Double Robustness in Right-Censored Longitudinal Models
• Rethinking the Inception Architecture for Computer Vision
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Deep Learning Face Attributes in the Wild
• Generating Sequences With Recurrent Neural Networks
• A Connection Between Score Matching and Denoising Autoencoders
• A local maximal inequality under uniform entropy
• Doubly Robust Estimation in Missing Data and Causal Inference Models
• Estimation of Regression Coefficients When Some Regressors are not Always Observed
• Acceleration of Stochastic Approximation by Averaging

Cited by

• Learning density ratios in causal inference using Bregman-Riesz regression
• Harnessing Synthetic Data from Generative AI for Statistical Inference

Discussions

• New paper on generative modeling of counterfactual distributions! We give a way to answer "what if" questions with generative models. For example: what would faces look like if they were all smiling? [bsky, 8 points, 1 comments]

Related

• De-Biasing Generative Models using Counterfactual Methods
• Counterfactual Generative Modeling with Variational Causal Inference
• Ablation Based Counterfactuals
• Conditional Generative Models for Counterfactual Explanations
• Improving Fairness and Mitigating MADness in Generative Models