Analysis of the momentum LMS algorithm

1990/01/01 by S. Roy, Susmita Roy, J.J. Shynk · 1 citation
Engineering · Computer Science · #Advanced Adaptive Filtering Techniques #Blind Source Separation Techniques #Neural Networks and Applications

paper · doi:10.1109/29.61535

Abstract

Several modifications of the well-known LMS algorithm have been proposed for improved operation. This work analyzes one such algorithm that corresponds to the standard LMS algorithm with an additional update term, parameterized by the scalar factor alpha where mod alpha mod>

Cited by

• The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning\n Rate Procedure For Least Squares

Related

• Better LMO-based Momentum Methods with Second-Order Information
• A Complete Transient Analysis for the Incremental LMS Algorithm
• AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training
• Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization
• AdaPM: a Partial Momentum Algorithm for LLM Training