2020/01/14 by Amedeo Roberto Esposito, Michael Gastpar, Esposito, Amedeo Roberto +3
Computer Science · Engineering · #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Neural Networks and Applications #Sparse and Compressive Sensing Techniques
paper · pdf · doi:10.48550/arxiv.2001.06399
openalex publication_date 2020/01/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
The aim of this work is to provide bounds connecting two probability measures\nof the same event using R 'enyi \α-Divergences and Sibson's\n\α-Mutual Information, a generalization of respectively the\nKullback-Leibler Divergence and Shannon's Mutual Information. A particular case\nof interest can be found when the two probability measures considered are a\njoint distribution and the corresponding product of marginals (representing the\nstatistically independent scenario). In this case, a bound using Sibson's\n\α-Mutual Information is retrieved, extending a result involving Maximal\nLeakage to general alphabets. These results have broad applications, from\nbounding the generalization error of learning algorithms to the more general\nframework of adaptive data analysis, provided that the divergences and/or\ninformation measures used are amenable to such an analysis ( it i.e., are\nrobust to post-processing and compose adaptively). The generalization error\nbounds are derived with respect to high-probability events but a corresponding\nbound on expected generalization error is also retrieved.\n