Information and estimation in Fokker-Planck channels

2017/02/13 by Wibisono, Andre, Jog, Varun, Loh, Po-Ling · 2 citations
#FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.1702.03656

Abstract

We study the relationship between information- and estimation-theoretic quantities in time-evolving systems. We focus on the Fokker-Planck channel defined by a general stochastic differential equation, and show that the time derivatives of entropy, KL divergence, and mutual information are characterized by estimation-theoretic quantities involving an appropriate generalization of the Fisher information. Our results vastly extend De Bruijn's identity and the classical I-MMSE relation.

Cited by

• Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolants
• Constrained Variational Inference via Safe Particle Flow

Related

• Differential Properties of Information in Jump-diffusion Channels
• Generalised Fisher Information in Defective Fokker-Planck Equations
• Sharp Decay of the Fisher Information for Degenerate Fokker-Planck Equations
• Dynamics of Uncertainty in Nonequilibrium Random Motion
• Random Dynamics, Entropy Production and Fisher Information