2010/11/30 by Domagoj Kuić, Domagoj Kuic, Paško Županović +3 · 1 citation
Mathematics · Physics and Astronomy · #Advanced Thermodynamics and Statistical Mechanics #Applied mathematics #Artificial intelligence #Computer science #Conditional probability distribution #Constraint (computer-aided design) #Entropy (arrow of time) #Entropy maximization #Inference #Information theory #Joint quantum entropy #Kullback–Leibler divergence #Mathematical analysis #Mathematical optimization #Mathematics #Maximization #Maximum entropy thermodynamics #Phase space #Physics #Principle of maximum entropy #Probability distribution #Quantum Mechanics and Applications #Quantum mechanics #Statistical Mechanics and Entropy #Statistical physics #Statistics #Upper and lower bounds #cond-mat.stat-mech
paper · pdf · doi:10.1007/s10701-011-9604-x
published as Found. Phys. 42, 319-339 (2012) · 25 pages
openalex publication_date 2011/11/08 · arxiv created 2012/02/20 · arxiv updated 2012/02/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
MaxEnt inference algorithm and information theory are relevant for the time evolution of macroscopic systems considered as problem of incomplete information. Two different MaxEnt approaches are introduced in this work, both applied to prediction of time evolution for closed Hamiltonian systems. The first one is based on Liouville equation for the conditional probability distribution, introduced as a strict microscopic constraint on time evolution in phase space. The conditional probability distribution is defined for the set of microstates associated with the set of phase space paths determined by solutions of Hamilton's equations. The MaxEnt inference algorithm with Shannon's concept of the conditional information entropy is then applied to prediction, consistently with this strict microscopic constraint on time evolution in phase space. The second approach is based on the same concepts, with a difference that Liouville equation for the conditional probability distribution is introduced as a macroscopic constraint given by a phase space average. We consider the incomplete nature of our information about microscopic dynamics in a rational way that is consistent with Jaynes' formulation of predictive statistical mechanics. Maximization of the conditional information entropy subject to this macroscopic constraint leads to a loss of correlation between the initial phase space paths and final microstates. Information entropy is the theoretic upper bound on the conditional information entropy, with the upper bound attained only in case of the complete loss of correlation. In this alternative approach to prediction of macroscopic time evolution, maximization of the conditional information entropy is equivalent to the loss of statistical correlation. In accordance with Jaynes, irreversibility appears as a consequence of gradual loss of information about possible microstates of the system.