2024/07/10 by Stuart Bartlett, Bartlett, Stuart J · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · #Adaptation and Self-Organizing Systems (nlin.AO) #Advanced Text Analysis Techniques #Biomedical Text Mining and Ontologies #FOS: Physical sciences #Statistical Mechanics (cond-mat.stat-mech) #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2407.07335
openalex publication_date 2024/07/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
I introduce a new approach to semantic information based upon the influence of erasure operations (interventions) upon distributions of a system's future trajectories through its phase space. Semantic (meaningful) information is distinguished from syntactic information by the property of having some intrinsic causal power on the future of a given system. As Shannon famously stated, syntactic information is a simple property of probability distributions (the elementary Shannon expression), or correlations between two subsystems and thus does not tell us anything about the meaning of a given message. Kolchinsky & Wolpert (2018) introduced a powerful framework for computing semantic information, which employs interventions upon the state of a system (either initial or dynamic) to erase syntactic information that might influence the viability of a subsystem (such as an organism in an environment). In this work I adapt this framework such that rather than using the viability of a subsystem, we simply observe the changes in future trajectories through a system's phase space as a result of informational interventions (erasures or scrambling). This allows for a more general formalisation of semantic information that does not assume a primary role for the viability of a subsystem (to use examples from Kolchinsky & Wolpert (2018), a rock, a hurricane, or a cell). Many systems of interest have a semantic component, such as a neural network, but may not have such an intrinsic connection to viability as living organisms or dissipative structures. Hence this simple approach to semantic information could be applied to any living, non-living or technological system in order to quantify whether a given quantity of syntactic information within it also has semantic or causal power.