2020/05/17 by Arnab Kumar Mondal, Arnab Bhattacharya, Mondal, Arnab Kumar +9
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #cs.IT #cs.LG #math.IT #stat.ML
paper · pdf · doi:10.48550/arxiv.2005.08226
Updated for UAI, 2020 camera-ready version
openalex publication_date 2020/05/17 · arxiv created 2020/07/23 · arxiv updated 2020/07/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Estimation of information theoretic quantities such as mutual information and its conditional variant has drawn interest in recent times owing to their multifaceted applications. Newly proposed neural estimators for these quantities have overcome severe drawbacks of classical kNN-based estimators in high dimensions. In this work, we focus on conditional mutual information (CMI) estimation by utilizing its formulation as a minmax optimization problem. Such a formulation leads to a joint training procedure similar to that of generative adversarial networks. We find that our proposed estimator provides better estimates than the existing approaches on a variety of simulated data sets comprising linear and non-linear relations between variables. As an application of CMI estimation, we deploy our estimator for conditional independence (CI) testing on real data and obtain better results than state-of-the-art CI testers.