2019/02/05 by Stefania Bellavia, Bellavia, Stefania, Nataša Krejić +3 · 2 citations
Computer Science · Engineering · #FOS: Mathematics #Machine Learning and Algorithms #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1902.01710
openalex publication_date 2019/02/05 · openalex created_date 2022/07/29 · openalex updated_date 2026/07/28
Convex and nonconvex finite-sum minimization arises in many scientific\ncomputing and machine learning applications. Recently, first-order and\nsecond-order methods where objective functions, gradients and Hessians are\napproximated by randomly sampling components of the sum have received great\nattention. We propose a new trust-region method which employs suitable\napproximations of the objective function, gradient and Hessian built via random\nsubsampling techniques. The choice of the sample size is deterministic and\nruled by the inexact restoration approach. We discuss local and global\nproperties for finding approximate first- and second-order optimal points and\nfunction evaluation complexity results. Numerical experience shows that the new\nprocedure is more efficient, in terms of overall computational cost, than the\nstandard trust-region scheme with subsampled Hessians.\n