2022/02/24 by Nicholas F. Marshall, Marshall, Nicholas F., Oscar Mickelin +1 · 1 citation
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Classical Analysis and ODEs (math.CA) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning and Algorithms #Numerical Analysis (math.NA) #Probability (math.PR) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2202.12224
openalex publication_date 2022/02/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We study how the learning rate affects the performance of a relaxed randomized Kaczmarz algorithm for solving A x ≈ b + ε, where A x =b is a consistent linear system and ε has independent mean zero random entries. We derive a learning rate schedule which optimizes a bound on the expected error that is sharp in certain cases; in contrast to the exponential convergence of the standard randomized Kaczmarz algorithm, our optimized bound involves the reciprocal of the Lambert-W function of an exponential.