2025/11/10 by Halyun Jeong, Jeong, Halyun, Palle E. T. Jorgensen +5
Computer Science · Decision Sciences · Engineering · #Stochastic Gradient Optimization Techniques #Advanced Bandit Algorithms Research #Sparse and Compressive Sensing Techniques
paper · pdf · doi:10.48550/arxiv.2511.07604
We present a variety of projection-based linear regression algorithms with a focus on modern machine-learning models and their algorithmic performance. We study the role of the relaxation parameter in generalized Kaczmarz algorithms and establish a priori regret bounds with explicit λ-dependence to quantify how much an algorithm's performance deviates from its optimal performance. A detailed analysis of relaxation parameter is also provided. Applications include: explicit regret bounds for the framework of Kaczmarz algorithm models, non-orthogonal Fourier expansions, and the use of regret estimates in modern machine learning models, including for noisy data, i.e., regret bounds for the noisy Kaczmarz algorithms. Motivated by machine-learning practice, our wider framework treats bounded operators (on infinite-dimensional Hilbert spaces), with updates realized as (block) Kaczmarz algorithms, leading to new and versatile results.