2018/03/09 by Andre Milzarek, Xiantao Xiao, Milzarek, Andre +7 · 1 citation
Engineering · Computer Science · Mathematics · #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #Advanced Optimization Algorithms Research
paper · pdf · doi:10.48550/arxiv.1803.03466
In this work, we present a globalized stochastic semismooth Newton method for\nsolving stochastic optimization problems involving smooth nonconvex and\nnonsmooth convex terms in the objective function. We assume that only noisy\ngradient and Hessian information of the smooth part of the objective function\nis available via calling stochastic first and second order oracles. The\nproposed method can be seen as a hybrid approach combining stochastic\nsemismooth Newton steps and stochastic proximal gradient steps. Two inexact\ngrowth conditions are incorporated to monitor the convergence and the\nacceptance of the semismooth Newton steps and it is shown that the algorithm\nconverges globally to stationary points in expectation. Moreover, under\nstandard assumptions and utilizing random matrix concentration inequalities, we\nprove that the proposed approach locally turns into a pure stochastic\nsemismooth Newton method and converges r-superlinearly with high probability.\nWe present numerical results and comparisons on \ℓ1-regularized logistic\nregression and nonconvex binary classification that demonstrate the efficiency\nof our algorithm.\n