1960/01/01 by Satosi Watanabé · 13 citations
Physics and Astronomy · Computer Science · #Statistical Mechanics and Entropy #Neural Networks and Applications #Computer science
paper · doi:10.1147/rd.41.0066
openalex publication_date 1960/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31
A set λ of stochastic variables, y1,y2, …, yn, is grouped into subsets, µ1, µ2, ..., µk. The correlation existing in λ with respect to the µ's is adequately expressed by an equation where S(ν) is the entropy function defined with reference to the variables y in subset ν. For a given λ, C becomes maximum when each µiconsists of only one variable, (n = k). The value C is then called the total correlation in λ, Ctot(λ). The present paper gives various theorems, according to which Ctot(λ) can be decomposed in terms of the partial correlations existing in subsets of λ, and of quantities derivable therefrom. The information-theoretical meaning of each decomposition is carefully explained. As illustrations, two problems are discussed at the end of the paper: (1) redundancy in geometrical figures in pattern recognition, and (2) randomization effect of shuffling cards marked “zero” or “one.”