Kullback-Leibler Divergence for the Normal-Gamma Distribution

2016/11/04 by Soch, Joram, Allefeld, Carsten
#FOS: Biological sciences #FOS: Mathematics #Neurons and Cognition (q-bio.NC) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.1611.01437

Abstract

We derive the Kullback-Leibler divergence for the normal-gamma distribution and show that it is identical to the Bayesian complexity penalty for the univariate general linear model with conjugate priors. Based on this finding, we provide two applications of the KL divergence, one in simulated and one in empirical data.

Related

• Computing the Kullback-Leibler Divergence between two Generalized Gamma Distributions
• Independent Gaussian Distributions Minimize the Kullback-Leibler (KL) Divergence from Independent Gaussian Distributions
• Distributions of the Kullback–Leibler divergence with applications
• A New Lower Bound for Kullback-Leibler Divergence Based on Hammersley-Chapman-Robbins Bound
• On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions