Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization

2014/12/10 by Saeed Ghadimi, Guanghui Lan, Hongchao Zhang · 64 citations
Computer Science · Engineering · Decision Sciences · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Risk and Portfolio Optimization

paper · doi:10.1007/s10107-014-0846-1

Citations

• A Stochastic Approximation Method

Cited by

• Scalable Policy Optimization for Networked Multi-Agent Reinforcement Learning with Continuous State-Action Spaces
• Data-Driven Games with Coherent Risk Measures
• Numerical methods in large-scale optimization: inexact oracle and primal-dual analysis
• Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization
• Convergence of adaptive algorithms for weakly convex constrained\n optimization
• WeMix: How to Better Utilize Data Augmentation
• A Polynomial Time Algorithm for Learning Halfspaces with Tsybakov Noise
• Escaping Saddle Points with Stochastically Controlled Stochastic Gradient Methods
• Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis
• Sample Efficient Policy Gradient Methods with Recursive Variance Reduction
• Embedding Samples Dispatching for Recommendation Model Training in Edge Environments
• A Parameter-Free Stochastic LineseArch Method (SLAM) for Minimizing Expectation Residuals
• A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimization
• Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization
• A Smooth Approximation Framework for Weakly Convex Optimization
• Closing the Approximation Gap of Partial AUC Optimization: A Tale of Two Formulations
• Asynchronous Decentralized Parallel Stochastic Gradient Descent
• Stochastic Non-convex Optimization with Strong High Probability Second-order Convergence
• Faster Gradient-Free Proximal Stochastic Methods for Nonconvex Nonsmooth Optimization
• On the Convergence and Stability of Distributed Sub-model Training
• Exploring Landscapes for Better Minima along Valleys
• Variable Smoothing Alternating Proximal Gradient Algorithm for Coupled Composite Optimization
• Stochastic Conditional Gradient++
• On the computation of equilibria in monotone and potential stochastic hierarchical games
• Efficient Optimization Algorithms for Robust Principal Component Analysis and Its Variants
• Zeroth-order optimisation on subsets of symmetric matrices with application to MPC tuning
• Continuous DR-submodular Maximization: Structure and Algorithms
• Stochastic Optimization for Non-convex Inf-Projection Problems
• A Unified Adaptive Tensor Approximation Scheme to Accelerate Composite Convex Optimization
• Complexity of finding near-stationary points of convex functions stochastically
• PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization
• Improved Learning Rates for Stochastic Optimization
• Stochastic Gradient Descent for Stochastic Doubly-Nonconvex Composite Optimization
• Coordinate-wise Control Variates for Deep Policy Gradients
• ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization
• A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization
• Natural Gradient VI: Guarantees for Non-Conjugate Models
• DualHash: A Stochastic Primal-Dual Algorithm with Theoretical Guarantee for Deep Hashing
• Stochastic Difference-of-Convex Optimization with Momentum
• Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis
• A General Family of Stochastic Proximal Gradient Methods for Deep Learning
• A Stochastic Derivative Free Optimization Method with Momentum
• SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Network
• Quantitative Convergence Analysis of Projected Stochastic Gradient Descent for Non-Convex Losses via the Goldstein Subdifferential
• Zeroth-Order Constrained Optimization from a Control Perspective via Feedback Linearization
• Finding Local Minima via Stochastic Nested Variance Reduction
• Inexact and Stochastic Gradient Optimization Algorithms with Inertia and Hessian Driven Damping
• Who to Trust? Aggregating Client Knowledge in Logit-Based Federated Learning
• On Tackling High-Dimensional Nonconvex Stochastic Optimization via Stochastic First-Order Methods with Non-smooth Proximal Terms and Variance Reduction
• Extragradient method with variance reduction for stochastic variational inequalities
• AdaGDA: Faster Adaptive Gradient Descent Ascent Methods for Minimax Optimization
• Momentum-based variance-reduced proximal stochastic gradient method for composite nonconvex stochastic optimization
• Asynchronous Variance-reduced Block Schemes for Composite Nonconvex Stochastic Optimization: Block-specific Steplengths and Adapted Batch-sizes
• Understanding the Effects of Data Parallelism and Sparsity on Neural\n Network Training
• Stochastic Proximal Methods for Non-Smooth Non-Convex Constrained Sparse Optimization
• Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
• Explainable Learning Rate Regimes for Stochastic Optimization
• Provable Smoothness Guarantees for Black-Box Variational Inference
• Zeroth-Order Stochastic Alternating Direction Method of Multipliers for Nonconvex Nonsmooth Optimization
• X-VFL: A New Vertical Federated Learning Framework with Cross Completion and Decision Subspace Alignment
• Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity
• Can SGD Handle Heavy-Tailed Noise?
• A Unified Framework for Stochastic Matrix Factorization via Variance Reduction
• Mini-batch stochastic Nesterov's smoothing method for constrained convex stochastic composite optimization