Learning Treatment Representations for Downstream Instrumental Variable Regression

2025/06/02 by Lin, Shiangyi, Hui Lan, Vasilis Syrgkanis +2
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine Learning in Healthcare #Methodology (stat.ME)

paper · pdf · doi:10.48550/arxiv.2506.02200

openalex publication_date 2025/06/02 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

Traditional instrumental variable (IV) estimators face a fundamental constraint: they can only accommodate as many endogenous treatment variables as available instruments. This limitation becomes particularly challenging in settings where the treatment is presented in a high-dimensional and unstructured manner (e.g. descriptions of patient treatment pathways in a hospital). In such settings, researchers typically resort to applying unsupervised dimension reduction techniques to learn a low-dimensional treatment representation prior to implementing IV regression analysis. We show that such methods can suffer from substantial omitted variable bias due to implicit regularization in the representation learning step. We propose a novel approach to construct treatment representations by explicitly incorporating instrumental variables during the representation learning process. Our approach provides a framework for handling high-dimensional endogenous variables with limited instruments. We demonstrate both theoretically and empirically that fitting IV models on these instrument-informed representations ensures identification of directions that optimize outcome prediction. Our experiments show that our proposed methodology improves upon the conventional two-stage approaches that perform dimension reduction without incorporating instrument information.

Citations

• Contrastive representations of high-dimensional, structured treatments
• Estimating Wage Disparities Using Foundation Models
• Counterfactual and Synthetic Control Method: Causal Inference with Instrumented Principal Component Analysis
• LABOR-LLM: Language-Based Occupational Representations with Large Language Models
• Identifiable Feature Learning for Spatial Data with Nonlinear ICA
• Learning Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity
• Identifying Representations for Intervention Extrapolation
• Source Condition Double Robust Inference on Functionals of Inverse Problems
• Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning
• Identifiability of latent-variable and structural-equation models: from linear to nonlinear
• Inference on Strongly Identified Functionals of Weakly Identified Functions
• Towards efficient representation identification in supervised learning
• Toward Causal Representation Learning
• Semiparametric Proximal Causal Inference
• GraphITE: Estimating Individual Effects of Graph-structured Treatments
• Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series
• Kernel Conditional Moment Test via Maximum Moment Restriction
• Causal Discovery with General Non-Linear Relationships Using Non-Linear\n ICA
• Kernel-based Tests for Joint Independence
• Independent component analysis: algorithms and applications

Related

• Learning Deep Features in Instrumental Variable Regression
• Learning Instrumental Variable from Data Fusion for Treatment Effect Estimation
• Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression
• Learning Representations of Instruments for Partial Identification of Treatment Effects
• Treatment Effects Inference with High-Dimensional Instruments and Control Variables