2019/07/02 by Pavlichin, Dmitri S., Quek, Yihui, Weissman, Tsachy
#Biological Physics (physics.bio-ph) #FOS: Computer and information sciences #FOS: Physical sciences #Information Theory (cs.IT) #Statistical Mechanics (cond-mat.stat-mech)
paper · doi:10.48550/arxiv.1907.01582
Biological systems use energy to maintain non-equilibrium distributions for long times, e.g. of chemical concentrations or protein conformations. What are the fundamental limits of the power used to "hold" a stochastic system in a desired distribution over states? We study the setting of an uncontrolled Markov chain Q altered into a controlled chain P having a desired stationary distribution. Thermodynamics considerations lead to an appropriately defined Kullback-Leibler (KL) divergence rate D(P||Q) as the cost of control, a setting introduced by Todorov, corresponding to a Markov decision process with mean log loss action cost. The optimal controlled chain P^* minimizes the KL divergence rate D(⋅||Q) subject to a stationary distribution constraint, and the minimal KL divergence rate lower bounds the power used. While this optimization problem is familiar from the large deviations literature, we offer a novel interpretation as a minimum "holding cost" and compute the minimizer P^* more explicitly than previously available. We state a version of our results for both discrete- and continuous-time Markov chains, and find nice expressions for the important case of a reversible uncontrolled chain Q, for a two-state chain, and for birth-and-death processes.