2017/03/17 by Linda Altieri, Altieri, Linda, Daniela Cocchi +3
Decision Sciences · Mathematics · Physics and Astronomy · #Advanced Statistical Methods and Models #FOS: Computer and information sciences #Methodology (stat.ME) #Multi-Criteria Decision Making #Statistical Mechanics and Entropy
paper · pdf · doi:10.48550/arxiv.1703.06001
openalex publication_date 2017/03/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The concept of entropy, firstly introduced in information theory, rapidly became popular in many applied sciences via Shannon's formula to measure the degree of heterogeneity among observations. A rather recent research field aims at accounting for space in entropy measures, as a generalization when the spatial location of occurrences ought to be accounted for. The main limit of these developments is that all indices are computed conditional on a chosen distance. This work follows and extends the route for including spatial components in entropy measures. Starting from the probabilistic properties of Shannon's entropy for categorical variables, it investigates the characteristics of the quantities known as residual entropy and mutual information, when space is included as a second dimension. This way, the proposal of entropy measures based on univariate distributions is extended to the consideration of bivariate distributions, in a setting where the probabilistic meaning of all components is well defined. As a direct consequence, a spatial entropy measure satisfying the additivity property is obtained, as global residual entropy is a sum of partial entropies based on different distance classes. Moreover, the quantity known as mutual information measures the information brought by the inclusion of space, and also has the property of additivity. A thorough comparative study illustrates the superiority of the proposed indices.