2024/01/06 by Peter Grassberger, Grassberger, Peter
Computer Science · #Bayesian Modeling and Causal Inference #FOS: Physical sciences #Neural Networks and Applications #Statistical Mechanics (cond-mat.stat-mech)
paper · pdf · doi:10.48550/arxiv.2401.03279
openalex publication_date 2024/01/06 · openalex created_date 2024/01/13 · openalex updated_date 2026/07/28
In a recent paper, Jurgens and Crutchfield [Phys. Rev. E \bf 104, 064107 (2021), called ``paper III" in the following] computed what they called the ``ambiguity rate" of hidden Markov processes, a concept supposedly introduced by Claude Shannon. This calculation was based on a ``mixed state" formalism introduced by them in J. Stat. Phys. \bf 183, 32 (2021) (``paper I"), and developed further in Chaos, \bf 31, 083114 (2021) (``paper II"). We point out that (i) ambiguity rates were \it not introduced by Shannon; (ii) their computations in paper III are wrong, because of an error made already in papers I and II; (iii) due to this error (a confusion between open sets and their closures), also many of the ``statistical complexity dimensions" computed in II are wrong; (iv) the ``mixed state" formalism of I is just the well known `forward algorithm' for hidden Markov models; (v) the `causal states' in `ε-machines' correspond in general to \it finite (as opposed to infinite, as often claimed) histories; and (vi) `ε-machines' are always countable, in contrast to frequent claims in the literature. In addition, we propose an alternative complexity measure for models where the forecasting complexity is infinite, and we point out that our results apply also beyond hidden Markov models.