Minimal Joint Entropy and Order-Preserving Couplings

2022/06/08 by Ma, Ya-Jing, Wang, Feng, Wu, Xian-Yuan +1 · 1 citation
#FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Probability (math.PR)

paper · doi:10.48550/arxiv.2206.03676

Abstract

This paper focuses on the extreme-value problem for Shannon entropy of the joint distribution with given marginals. It is proved that the minimum-entropy coupling must be of order-preserving, while the maximum-entropy coupling coincides with the independent one. Note that in this sense, we interpret entropy as a measure of system disorder.

Cited by

• An Explicit Description of Extreme Points of the Set of Couplings with Given Marginals: with Application to Minimum-Entropy Coupling Problems

Related

• On the uniqueness of the coupled entropy
• Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations
• Minimum--Entropy Couplings and their Applications
• Coupled Entropy: A Goldilocks Generalization for Complex Systems
• Minimizing and Maximizing the Shannon Entropy for Fixed Marginals