Exact Computation of Kullback-Leibler Distance for Hidden Markov Trees and Models

2011/12/14 by Vittorio Perduca, Perduca, Vittorio, Grégory Nuel +1
Computer Science · Physics and Astronomy · #Data Management and Algorithms #Bayesian Methods and Mixture Models #Complex Network Analysis Techniques

paper · pdf · doi:10.48550/arxiv.1112.3257

Abstract

We suggest new recursive formulas to compute the exact value of the Kullback-Leibler distance (KLD) between two general Hidden Markov Trees (HMTs). For homogeneous HMTs with regular topology, such as homogeneous Hidden Markov Models (HMMs), we obtain a closed-form expression for the KLD when no evidence is given. We generalize our recursive formulas to the case of HMMs conditioned on the observable variables. Our proposed formulas are validated through several numerical examples in which we compare the exact KLD value with Monte Carlo estimations.

Citations

• A tutorial on hidden Markov models and selected applications in speech recognition

Related

• On Computing the Total Variation Distance of Hidden Markov Models
• Kullback-Leibler Divergence and Akaike Information Criterion in General Hidden Markov Models
• A Distance for HMMs based on Aggregated Wasserstein Metric and State Registration
• Spectral dimensionality reduction for HMMs
• Comparing Labeled Markov Chains: A Cantor-Kantorovich Approach