Model-Powered Conditional Independence Test

2017/09/18 by Rajat Sen, Ananda Theertha Suresh, Sen, Rajat +6 · 3 citations
Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.1709.06138

openalex publication_date 2017/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider the problem of non-parametric Conditional Independence testing (CI testing) for continuous random variables. Given i.i.d samples from the joint distribution f(x,y,z) of continuous random vectors X,Y and Z, we determine whether X ⊥ Y | Z. We approach this by converting the conditional independence test into a classification problem. This allows us to harness very powerful classifiers like gradient-boosted trees and deep neural networks. These models can handle complex probability distributions and allow us to perform significantly better compared to the prior state of the art, for high-dimensional CI testing. The main technical challenge in the classification problem is the need for samples from the conditional product distribution f^CI(x,y,z) = f(x|z)f(y|z)f(z) -- the joint distribution if and only if X ⊥ Y | Z. -- when given access only to i.i.d. samples from the true joint distribution f(x,y,z). To tackle this problem we propose a novel nearest neighbor bootstrap procedure and theoretically show that our generated samples are indeed close to f^CI in terms of total variational distance. We then develop theoretical results regarding the generalization bounds for classification for our problem, which translate into error bounds for CI testing. We provide a novel analysis of Rademacher type classification bounds in the presence of non-i.i.d near-independent samples. We empirically validate the performance of our algorithm on simulated and real datasets and show performance gains over previous methods.

Cited by

• Toward Scalable and Valid Conditional Independence Testing with Spectral Representations
• On the Hardness of Conditional Independence Testing In Practice
• Efficient Ensemble Conditional Independence Test Framework for Causal Discovery

Related

• Mimic and Classify : A meta-algorithm for Conditional Independence Testing
• Conditional independence testing based on a nearest-neighbor estimator of conditional mutual information
• Conditional Independence Testing Using Exchangeable Pairs
• Minimax Optimal Conditional Independence Testing
• Conditional Diffusion Models Based Conditional Independence Testing