1998/01/28 by Renato Vicente, Osame Kinouchi, Nestor Caticha · 1 citation
Physics and Astronomy · #cond-mat.dis-nn #cond-mat.stat-mech
published as Machine Learning 32 179-201 (1998) · 24 pages, 8 figures, to appear in Machine Learning Journal
arxiv created 1998/01/28 · arxiv updated 2009/11/30
We review the application of Statistical Mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feed-forward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. The constructive variational method also suggests a learning algorithm. It depends, however, on some unavailable quantities, such as the present performance of the student. The construction of estimators for these quantities permits the implementation of a very effective, highly adaptive algorithm. Several other algorithms are also studied for comparison with the optimal bound and the adaptive algorithm, for different types of time evolution of the rule.