2022/02/28 by Parikshit Gopalan, Nina Narodytska, Gopalan, Parikshit +7
Computer Science · Neuroscience · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Psychology of Moral and Emotional Judgment
paper · pdf · doi:10.48550/arxiv.2202.13576
openalex publication_date 2022/02/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Estimating the Kullback-Leibler (KL) divergence between two distributions given samples from them is well-studied in machine learning and information theory. Motivated by considerations of multi-group fairness, we seek KL divergence estimates that accurately reflect the contributions of sub-populations to the overall divergence. We model the sub-populations coming from a rich (possibly infinite) family C of overlapping subsets of the domain. We propose the notion of multi-group attribution for C, which requires that the estimated divergence conditioned on every sub-population in C satisfies some natural accuracy and fairness desiderata, such as ensuring that sub-populations where the model predicts significant divergence do diverge significantly in the two distributions. Our main technical contribution is to show that multi-group attribution can be derived from the recently introduced notion of multi-calibration for importance weights [HKRR18, GRSW21]. We provide experimental evidence to support our theoretical results, and show that multi-group attribution provides better KL divergence estimates when conditioned on sub-populations than other popular algorithms.