2006/06/27 by M. Rezaeian, Rezaeian, Mohammad
Computer Science · Engineering · #Bayesian Methods and Mixture Models #Control Systems and Identification #FOS: Computer and information sciences #Information Theory (cs.IT) #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.cs/0606114
openalex publication_date 2006/06/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider a pair of correlated processes Zn and Sn (two sided), where the former is observable and the later is hidden. The uncertainty in the estimation of Zn upon its finite past history is H(Zn|Z0n-1), and for estimation of Sn upon this observation is H(Sn|Z0n-1), which are both sequences of n. The limits of these sequences (and their existence) are of practical and theoretical interest. The first limit, if exists, is the entropy rate. We call the second limit the estimation entropy. An example of a process jointly correlated to another one is the hidden Markov process. It is the memoryless observation of the Markov state process where state transitions are independent of past observations. We consider a new representation of hidden Markov process using iterated function system. In this representation the state transitions are deterministically related to the process. This representation provides a unified framework for the analysis of the two limiting entropies for this process, resulting in integral expressions for the limits. This analysis shows that under mild conditions the limits exist and provides a simple method for calculating the elements of the corresponding sequences.