Penalty Methods with Stochastic Approximation for Stochastic Nonlinear Programming

2013/12/10 by Xiao Wang, Wang, Xiao, Shiqian Ma +3 · 3 citations
Computer Science · Decision Sciences · Mathematics · #Advanced Optimization Algorithms Research #FOS: Mathematics #Optimization and Control (math.OC) #Risk and Portfolio Optimization #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1312.2690

openalex publication_date 2013/12/10 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28

Abstract

In this paper, we propose a class of penalty methods with stochastic approximation for solving stochastic nonlinear programming problems. We assume that only noisy gradients or function values of the objective function are available via calls to a stochastic first-order or zeroth-order oracle. In each iteration of the proposed methods, we minimize an exact penalty function which is nonsmooth and nonconvex with only stochastic first-order or zeroth-order information available. Stochastic approximation algorithms are presented for solving this particular subproblem. The worst-case complexity of calls to the stochastic first-order (or zeroth-order) oracle for the proposed penalty methods for obtaining an ε-stochastic critical point is analyzed.

Citations

• Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimization
• Numerical optimization

Cited by

• Adaptive directional decomposition methods for nonconvex constrained optimization
• ASPEN: An Additional Sampling Penalty Method for Finite-Sum Optimization Problems with Nonlinear Equality Constraints
• Single-loop O(ε^-3) stochastic smoothing algorithms for nonsmooth Riemannian optimization

Related

• Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming
• Stochastic Approximation for Expectation Objective and Expectation Inequality-Constrained Nonconvex Optimization
• Stochastic first-order methods: non-asymptotic and computer-aided analyses via potential functions
• Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
• Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization