Entropy and Correlation: Some Comments

1987/01/01 by Tarald O. Kvalseth, Tarald O. Kvålseth · 256 citations
Computer Science · Mathematics · #Applied mathematics #Computer science #Correlation #Data mining #Econometrics #Entropy (arrow of time) #Geometry #Mathematics #Measure (data warehouse) #Neural Networks and Applications #Physics #Sample size determination #Statistical inference #Statistical physics #Statistics #Thermodynamics

paper · doi:10.1109/tsmc.1987.4309069

published in IEEE Transactions on Systems Man and Cybernetics 17(3), 517-519 (Institute of Electrical and Electronics Engineers)

openalex publication_date 1987/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

For measuring the degree of association or correlation between two nominal variables, a measure based on informational entropy is presented as being preferable to that proposed recently by Horibe [1]. Asymptotic developments are also presented that may be used for making approximate statistical inferences about the population measure when the sample size is reasonably large. The use of this methodology is illustrated using a numerical example.

Cited by

• Belief Network Analysis: A Relational Approach to Understanding the Structure of Attitudes
• Social Comparison without Explicit Inference of Others' Reward Values: A Constructive Approach Using a Probabilistic Generative Model
• Understanding partition comparison indices based on counting object pairs
• Mode-Aware Non-Linear Tucker Autoencoder for Tensor-based Unsupervised Learning
• Normalised clustering accuracy: An asymmetric external cluster validity measure
• Measuring Information Leakage in Website Fingerprinting Attacks and Defenses
• Unveiling 3D Ocean Biogeochemical Provinces in the North Atlantic: A Systematic Comparison and Validation of Clustering Methods
• Information-theoretic Measures for Objective Evaluation of Classifications
• A non-reference image fusion metric based on mutual information of image features
• Locality Preserving Projections for Grassmann manifold
• Characterizing and Comparing External Measures for the Assessment of Cluster Analysis and Community Detection
• Core–periphery structure in directed networks
• Comparison of module detection algorithms in protein networks and investigation of the biological meaning of predicted modules. [europepmc]
• Core-periphery structure in directed networks. [europepmc]
• Inter-protein residue covariation information unravels physically interacting protein dimers. [europepmc]
• Cryogenian Origin and Subsequent Diversification of the Plant Cell-Wall Enzyme XTH Family. [europepmc]
• A fractal scaling analysis of the SARS-CoV-2 genome sequence. [europepmc]
• Image Registration in Longitudinal Bone Assessment Using Computed Tomography. [europepmc]
• Enhancing Exchange-Traded Fund Price Predictions: Insights from Information-Theoretic Networks and Node Embeddings. [europepmc]
• Hierarchical path planning from speech instructions with spatial concept-based topometric semantic mapping. [europepmc]
• Fusion of EEG and EMG signals for detecting pre-movement intention of sitting and standing in healthy individuals and patients with spinal cord injury. [europepmc]
• Perceptual Complexity as Normalized Shannon Entropy. [europepmc]
• Null models for comparing information decomposition across complex systems. [europepmc]
• deepBreaks identifies and prioritizes genotype-phenotype associations using machine learning. [europepmc]

Related

• Entropy rates in the dimension theory of self-similar measures
• The Boltzmann/Shannon entropy as a measure of correlation
• A Note on Mathai's Entropy Measure
• Physics behind the minimum of relative entropy measures for correlations
• Entropy measures and their applications: A comprehensive review