2018/08/08 by L. A. Prashanth, Prashanth L A, Shalabh Bhatnagar +10
Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · Mathematics · #3D Shape Modeling and Analysis #Diffusion and Search Dynamics #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Point processes and geometric inequalities #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference #cs.LG #math.OC
paper · pdf · doi:10.48550/arxiv.1808.02871
openalex publication_date 2018/08/08 · arxiv created 2019/03/28 · arxiv updated 2019/03/29 · openalex created_date 2022/08/04 · openalex updated_date 2026/07/28
We introduce deterministic perturbation schemes for the recently proposed random directions stochastic approximation (RDSA) [17], and propose new first-order and second-order algorithms. In the latter case, these are the first second-order algorithms to incorporate deterministic perturbations. We show that the gradient and/or Hessian estimates in the resulting algorithms with deterministic perturbations are asymptotically unbiased, so that the algorithms are provably convergent. Furthermore, we derive convergence rates to establish the superiority of the first-order and second-order algorithms, for the special case of a convex and quadratic optimization problem, respectively. Numerical experiments are used to validate the theoretical results.