2021/07/31 by James Fullwood, Arthur J. Parzygnat
Computer Science · Mathematics · #cs.IT #math.CT #math.IT #math.PR #msc:18A05 #msc:62F15 #msc:94A17
paper · pdf · doi:10.3390/e23081021
published as Entropy 2021, 23(8), 1021 · 31 pages; Typos fixed in Defn 2.12 and the proofs of Prop 4.2 iii) and Prop 6.8 (numbering scheme differs from published version)
arxiv created 2021/12/22 · arxiv updated 2021/12/23
We provide a stochastic extension of the Baez-Fritz-Leinster characterization of the Shannon information loss associated with a measure-preserving function. This recovers the conditional entropy and a closely related information-theoretic measure that we call conditional information loss. Although not functorial, these information measures are semi-functorial, a concept we introduce that is definable in any Markov category. We also introduce the notion of an entropic Bayes' rule for information measures, and we provide a characterization of conditional entropy in terms of this rule.