2017/10/13 by Arman Rahimzamani, Rahimzamani, Arman, Sreeram Kannan +1 · 1 citation
Computer Science · Mathematics · #Bayesian Modeling and Causal Inference #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Statistical Methods and Inference
paper · pdf · doi:10.48550/arxiv.1710.05012
openalex publication_date 2017/10/13 · openalex created_date 2022/09/11 · openalex updated_date 2026/07/28
The conditional mutual information I(X;Y|Z) measures the average information\nthat X and Y contain about each other given Z. This is an important primitive\nin many learning problems including conditional independence testing, graphical\nmodel inference, causal strength estimation and time-series problems. In\nseveral applications, it is desirable to have a functional purely of the\nconditional distribution pY|X,Z rather than of the joint distribution\npX,Y,Z. We define the potential conditional mutual information as the\nconditional mutual information calculated with a modified joint distribution\npY|X,Z qX,Z, where qX,Z is a potential distribution, fixed airport. We\ndevelop K nearest neighbor based estimators for this functional, employing\nimportance sampling, and a coupling trick, and prove the finite k consistency\nof such an estimator. We demonstrate that the estimator has excellent practical\nperformance and show an application in dynamical system inference.\n