2024/08/14 by Zhaolu Liu, Liu, Zhaolu, Mauricio Barahona +3 · 3 citations
Computer Science · Physics and Astronomy · #Advanced Thermodynamics and Statistical Mechanics #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (stat.ML) #Quantum Computing Algorithms and Architecture #Quantum Mechanics and Applications
paper · pdf · doi:10.48550/arxiv.2408.07533
openalex publication_date 2024/08/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Traditional measures based solely on pairwise associations often fail to capture the complex statistical structure of multivariate data. Existing approaches for identifying information shared among d>3 variables are frequently computationally intractable, asymmetric with respect to a target variable, or unable to account for all the ways in which the joint probability distribution can be factorised. Here we present a systematic framework based on lattice theory to derive higher-order information-theoretic measures for multivariate data. Our construction uses lattice and operator function pairs, whereby an operator function is applied over a lattice that represents the algebraic relationships among variables. We show that many commonly used measures can be derived within this framework, yet they fail to capture all interactions for d>3, either because they are defined on restricted sublattices, or because the use of the KL divergence as an operator function, a typical choice, leads to undesired disregard of groups of interactions. To fully characterise all interactions among d variables, we introduce the Streitberg Information, which is defined over the full partition lattice and uses generalised divergences (beyond KL) as operator functions. We validate the Streitberg Information on synthetic data, and illustrate its application in detecting complex interactions among stocks, decoding neural signals, and performing feature selection in machine learning.