2005/03/21 by Alexander Plakhov, Plakhov, Alexander, Cruz, Pedro
Computer Science · Engineering · #62L20 #FOS: Mathematics #Neural Networks and Applications #Probability (math.PR) #Sparse and Compressive Sensing Techniques #Statistics Theory (math.ST) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.math/0503434
openalex publication_date 2005/03/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
An algorithm of searching a zero of an unknown undimensional function is considered, measured at a point x with some error. The step sizes are random positive values and are calculated according to the rule: if two consecutive iterations are in same direction step is multiplied by u>1, otherwise, it is multiplied by 01, divergence. Due to the multiplicative rule of updating of the step, it is natural to expect that the sequence converges rapidly: like a geometric progression (if convergence takes place), but the limit value may not coincide with, but instead, approximates one of zeros of the function. By adjusting the parameters u and d, one can reach necessary precision of approximation; higher precision is obtained at the expense of lower convergence rate.