Stochastic Approximation for Expectation Objective and Expectation Inequality-Constrained Nonconvex Optimization

2023/07/06 by Francisco Facchinei, Facchinei, Francisco, Vyacheslav Kungurtsev +1 · 2 citations
Computer Science · Decision Sciences · Mathematics · #FOS: Mathematics #Optimization and Control (math.OC) #Risk and Portfolio Optimization #Statistical Methods and Inference #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2307.02943

openalex publication_date 2023/07/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Stochastic Approximation has been a prominent set of tools for solving problems with noise and uncertainty. Increasingly, it becomes important to solve optimization problems wherein there is noise in both a set of constraints that a practitioner requires the system to adhere to, as well as the objective, which typically involves some empirical loss. We present the first stochastic approximation approach for solving this class of problems using the Ghost framework of incorporating penalty functions for analysis of a sequential convex programming approach together with a Monte Carlo estimator of nonlinear maps. We provide almost sure convergence guarantees and demonstrate the performance of the procedure on some representative examples.

Cited by

• humancompatible.train: Implementing Optimization Algorithms for Stochastically-Constrained Stochastic Optimization Problems
• Benchmarking Stochastic Approximation Algorithms for Fairness-Constrained Training of Deep Neural Networks

Related

• Algorithms for stochastic optimization with functional or expectation constraints
• Optimal Convergence for Stochastic Optimization with Multiple Expectation Constraints
• Solving Stochastic Optimization with Expectation Constraints Efficiently by a Stochastic Augmented Lagrangian-Type Algorithm
• A stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs
• Stochastic Approximation Proximal Method of Multipliers for Convex Stochastic Programming