2005/03/21 by Alexander Plakhov, Plakhov, Alexander, Pedro Cruz +1
Computer Science · Engineering · Mathematics · #62L20 #FOS: Mathematics #Neural Networks and Applications #Probability (math.PR) #Sparse and Compressive Sensing Techniques #Statistics Theory (math.ST) #Stochastic Gradient Optimization Techniques #math.PR #math.ST #msc:62L20 #stat.TH
paper · pdf · doi:10.48550/arxiv.math/0503434
arxiv created 2005/03/21 · openalex publication_date 2005/03/21 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
An algorithm of searching a zero of an unknown undimensional function is considered, measured at a point x with some error. The step sizes are random positive values and are calculated according to the rule: if two consecutive iterations are in same direction step is multiplied by u>1, otherwise, it is multiplied by 0<d<1. The function may have one or more zeros; the random values are independent and identically distributed, with zero mean and finite variance. Under some additional assumptions on the conditions on the two parameters u and d almost sure convergence of the sequence as well as under some conditions is guaranteed almost sure divergence. In particular, if the error distribuition as median 0 and zero probability for particular poinst then it is established that for ud<1, convergence takes place, and for ud>1, divergence. Due to the multiplicative rule of updating of the step, it is natural to expect that the sequence converges rapidly: like a geometric progression (if convergence takes place), but the limit value may not coincide with, but instead, approximates one of zeros of the function. By adjusting the parameters u and d, one can reach necessary precision of approximation; higher precision is obtained at the expense of lower convergence rate.