2008/04/10 by Andre Martins, Martins, Andre, Pedro Aguiar +3 · 1 citation
Computer Science · Mathematics · #94A15 #94A17 #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Statistics Theory (math.ST) #cs.IT #math.IT #math.ST #msc:94A15 #msc:94A17 #stat.TH
paper · pdf · doi:10.48550/arxiv.0804.1653
Submitted to the IEEE Transactions on Information Theory
arxiv created 2008/04/10 · arxiv updated 2009/12/01
Convexity is a key concept in information theory, namely via the many implications of Jensen's inequality, such as the non-negativity of the Kullback-Leibler divergence (KLD). Jensen's inequality also underlies the concept of Jensen-Shannon divergence (JSD), which is a symmetrized and smoothed version of the KLD. This paper introduces new JSD-type divergences, by extending its two building blocks: convexity and Shannon's entropy. In particular, a new concept of q-convexity is introduced and shown to satisfy a Jensen's q-inequality. Based on this Jensen's q-inequality, the Jensen-Tsallis q-difference is built, which is a nonextensive generalization of the JSD, based on Tsallis entropies. Finally, the Jensen-Tsallis q-difference is charaterized in terms of convexity and extrema.