2014/10/18 by M. Ashok Kumar, Kumar, M. Ashok, Rajesh Sundaresan +1
Physics and Astronomy · #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Statistical Mechanics and Entropy #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.1410.4931
openalex publication_date 2014/10/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We study minimization of a parametric family of relative entropies, termed relative α-entropies (denoted \mathscrIα(P,Q)). These arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative α-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimization of \mathscrIα(P,Q) over the first argument on a set of probability distributions that constitutes a linear family is studied. Such a minimization generalizes the maximum Rényi or Tsallis entropy principle. The minimizing probability distribution (termed \mathscrIα-projection) for a linear family is shown to have a power-law.