A simple measure of conditional dependence

2019/10/27 by Mona Azadkia, Sourav Chatterjee, Azadkia, Mona +1 · 1 voice · 22 citations
Computer Science · Mathematics · #Advanced Statistical Methods and Models #Applied mathematics #Bayesian Modeling and Causal Inference #Combinatorics #Computer science #Conditional expectation #Conditional independence #Data mining #Generalization #Independence (probability theory) #Limit (mathematics) #Mathematical analysis #Mathematics #Measure (data warehouse) #Random variable #Simple (philosophy) #Statistic #Statistical Methods and Inference #Statistical hypothesis testing #Statistics #Test statistic #cs.IT #math.IT #math.PR #math.ST #msc:62G05 #msc:62H20 #stat.ME #stat.TH

paper · pdf · doi:10.1214/21-aos2073

published in The Annals of Statistics 49(6) (Institute of Mathematical Statistics) · 41 pages, 2 tables. Final version. To appear in Ann. Statist. An R package is available at https://CRAN.R-project.org/package=FOCI

arxiv created 2021/03/28 · arxiv updated 2021/03/30 · openalex publication_date 2021/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

We propose a coefficient of conditional dependence between two random variables Y and Z given a set of other variables X1,…,Xp, based on an i.i.d. sample. The coefficient has a long list of desirable properties, the most important of which is that under absolutely no distributional assumptions, it converges to a limit in [0,1], where the limit is 0 if and only if Y and Z are conditionally independent given X1,…,Xp, and is 1 if and only if Y is equal to a measurable function of Z given X1,…,Xp. Moreover, it has a natural interpretation as a nonlinear generalization of the familiar partial R2 statistic for measuring conditional dependence by regression. Using this statistic, we devise a new variable selection algorithm, called Feature Ordering by Conditional Independence (FOCI), which is model-free, has no tuning parameters, and is provably consistent under sparsity assumptions. A number of applications to synthetic and real data sets are worked out.

Cited by

• Fast Wasserstein rates for estimating probability distributions of probabilistic graphical models
• Covariate-adjusted statistical dependence representation through partial copulas: bounds and new insights
• An ordering for the strength of functional dependence
• Super doubly robust and efficient estimator for informative covariate censoring
• On Azadkia-Chatterjee's conditional dependence coefficient
• MINERVA: Mutual Information Neural Estimation for Supervised Feature Selection
• One-shot Conditional Sampling: MMD meets Nearest Neighbors
• Measuring Association on Topological Spaces Using Kernels and Geometric Graphs
• Top-k Feature Importance Ranking
• Conditional Independence Testing Using Exchangeable Pairs
• Quantifying and testing dependence to categorical variables
• Measuring General Associations in Time Series: An Adaptation and Empirical Evaluation of the CODEC Coefficient in Determining Autoregressive Dynamics
• On computing and the complexity of computing higher-order U-statistics, exactly
• Bias correction for Chatterjee's graph-based correlation coefficient
• Coverage correlation: detecting singular dependencies between random variables
• The exact region and an inequality between Chatterjee's and Spearman's rank correlations
• A dimension reduction for extreme types of directed dependence
• Score-based Generative Modeling for Conditional Independence Testing
• Correlations with tailored extremal properties
• A new measure of dependence: Integrated R²
• Measures of association for approximating copulas
• Measuring Feature-Label Dependence Using Projection Correlation Statistic

Discussions

• Canonical correlation or the RV coefficient if the variables are in groups or the FOCI method by Azadkia and Chaterjee . That paper (arxiv.org/abs/1910.12327) also references many such coefficients.. [bsky, 1 points, 1 comments]

Related

• A new measure of dependence: Integrated R²
• Quantifying and estimating dependence via sensitivity of conditional distributions
• Conditional Dependence via U-Statistics Pruning
• A New Measure of Conditional Dependence
• A Distribution Free Conditional Independence Test with Applications to Causal Discovery