On-line learning with restricted training sets: An exactly solvable case

1998/11/16 by H. C. Rae, H C Rae, P. Sollich +3
Computer Science · Physics and Astronomy · #Machine Learning and Algorithms #Neural Networks and Applications #Stochastic Gradient Optimization Techniques #cond-mat.dis-nn #cond-mat.stat-mech

paper · pdf · doi:10.1088/0305-4470/32/18/308

published as J. Phys. A: Math. Gen., 32: 3321-3339, 1999 · 19 pages, eps figures included, uses epsfig macro

arxiv created 1998/11/16 · openalex publication_date 1999/01/01 · arxiv updated 2009/11/30 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/30

Abstract

We solve the dynamics of on-line Hebbian learning in large perceptrons exactly, for the regime where the size of the training set scales linearly with the number of inputs. We consider both noiseless and noisy teachers. Our calculation cannot be extended to non-Hebbian rules, but the solution provides a convenient and welcome benchmark with which to test more general and advanced theories for solving the dynamics of learning with restricted training sets.

Citations

• Learning by on-line gradient descent

Related

• Dynamics of Learning with Restricted Training Sets II: Tests and Applications
• On-line learning in a discrete state space
• Statistical Features in Learning
• A Simple Perceptron that Learns Non-Monotonic Rules
• Dynamics of Supervised Learning with Restricted Training Sets