2022/06/28 by Thierrin, Ferenc Cole, Alajaji, Fady, Linder, Tamás
#FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG)
paper · doi:10.48550/arxiv.2206.14329
The Rényi cross-entropy measure between two distributions, a generalization of the Shannon cross-entropy, was recently used as a loss function for the improved design of deep learning generative adversarial networks. In this work, we examine the properties of this measure and derive closed-form expressions for it when one of the distributions is fixed and when both distributions belong to the exponential family. We also analytically determine a formula for the cross-entropy rate for stationary Gaussian processes and for finite-alphabet Markov sources.