2013/04/20 by Michael P. Friedlander, Friedlander, Michael P., Gabriel Goh +1
Mathematics · #FOS: Mathematics #Optimization and Control (math.OC) #math.OC
paper · pdf · doi:10.48550/arxiv.1304.5586
arxiv created 2014/01/08 · arxiv updated 2014/01/09
Stochastic-approximation gradient methods are attractive for large-scale convex optimization because they offer inexpensive iterations. They are especially popular in data-fitting and machine-learning applications where the data arrives in a continuous stream, or it is necessary to minimize large sums of functions. It is known that by appropriately decreasing the variance of the error at each iteration, the expected rate of convergence matches that of the underlying deterministic gradient method. Conditions are given under which this happens with overwhelming probability.