2012/05/30 by Bao-Gang Hu, Hu, Bao-Gang, Hong-Jie Xing +1
Computer Science · Physics and Astronomy · #Anomaly Detection Techniques and Applications #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #Information Theory (cs.IT) #Statistical Mechanics and Entropy
paper · pdf · doi:10.48550/arxiv.1205.6602
openalex publication_date 2012/05/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The existing upper and lower bounds between entropy and error probability are mostly derived from the inequality of the entropy relations, which could introduce approximations into the analysis. We derive analytical bounds based on the closed-form solutions of conditional entropy without involving any approximation. Two basic types of classification errors are investigated in the context of binary classification problems, namely, Bayesian and non-Bayesian errors. We theoretically confirm that Fano's lower bound is an exact lower bound for any types of classifier in a relation diagram of "error probability vs. conditional entropy". The analytical upper bounds are achieved with respect to the minimum prior probability, which are tighter than Kovalevskij's upper bound.