2024/09/03 by Avishek Das, Pieter Rein ten Wolde, Das, Avishek +1 · 3 citations
Decision Sciences · #Biological Physics (physics.bio-ph) #FOS: Biological sciences #FOS: Computer and information sciences #FOS: Physical sciences #Information Theory (cs.IT) #Molecular Networks (q-bio.MN) #Probabilistic and Robust Engineering Design #Soft Condensed Matter (cond-mat.soft) #Statistical Mechanics (cond-mat.stat-mech)
paper · pdf · doi:10.48550/arxiv.2409.01650
openalex publication_date 2024/09/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The ability to quantify the directional flow of information is vital to understanding natural systems and designing engineered information-processing systems. A widely used measure to quantify this information flow is the transfer entropy. However, until now, this quantity could only be obtained in dynamical models using approximations that are typically uncontrolled. Here we introduce a computational algorithm called Transfer Entropy-Path Weight Sampling (TE-PWS), which makes it possible, for the first time, to quantify the transfer entropy and its variants exactly for any stochastic model, including those with multiple hidden variables, nonlinearity, transient conditions, and feedback. By leveraging techniques from polymer and path sampling, TE-PWS efficiently computes the transfer entropy as a Monte-Carlo average over signal trajectory space. We use our exact technique to demonstrate that commonly used approximate methods to compute transfer entropies incur large systematic errors and high computational costs. As an application, we use TE-PWS in linear and nonlinear systems to reveal how transfer entropy can overcome naive applications of the data processing inequality in the presence of feedback.