Fluctuation-dissipation theorem and models of learning

2004/02/29 by Ilya Nemenman
Biochemistry, Genetics and Molecular Biology · Computer Science · Physics and Astronomy · #q-bio.NC #cs.LG #nlin.AO #physics.data-an

paper · pdf

published as Neural Comp. 17 (9): 2006-2033 SEP 2005 · 23 pages, 1 figure; manuscript restructured following reviewers' suggestions; references added; misprints corrected

arxiv created 2004/10/26 · arxiv updated 2009/12/01

Abstract

Advances in statistical learning theory have resulted in a multitude of different designs of learning machines. But which ones are implemented by brains and other biological information processors? We analyze how various abstract Bayesian learners perform on different data and argue that it is difficult to determine which learning-theoretic computation is performed by a particular organism using just its performance in learning a stationary target (learning curve). Basing on the fluctuation-dissipation relation in statistical physics, we then discuss a different experimental setup that might be able to solve the problem.

Related

• Non-equilibrium physics: from spin glasses to machine and neural learning
• Fluctuation-learning relationship in neural networks
• Learning principle and mathematical realization of the learning mechanism in the brain
• The principles of adaptation in organisms and machines II: Thermodynamics of the Bayesian brain
• A Theory of Machine Learning