Solving Stochastic Optimization by Newton-type methods with Dimension-Adaptive Sparse Grid Quadrature

2022/02/21 by Zhou YuanCheng, Zhou, Yuancheng
Computer Science · Mathematics · #65Kxx #Advanced Multi-Objective Optimization Algorithms #FOS: Mathematics #Numerical Analysis (math.NA) #Statistical Methods and Inference #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2202.10364

openalex publication_date 2022/02/21 · openalex created_date 2022/04/03 · openalex updated_date 2026/07/28

Abstract

Stochastic optimisation problems minimise expectations of random cost functions. We use 'optimise then discretise' method to solve stochastic optimisation. In our approach, accurate quadrature methods are required to calculate the objective, gradient or Hessian which are in fact integrals. We apply the dimension-adaptive sparse grid quadrature to approximate these integrals when the problem is high dimensional. Dimension-adaptive sparse grid quadrature shows high accuracy and efficiency in computing an integral with a smooth integrand. It is a kind of generalisation of the classical sparse grid method, which refines different dimensions according to their importance. We show that the dimension-adaptive sparse grid quadrature has better performance in the optimise then discretise' method than the 'discretise then optimise' method.

Related

• Stochastic quasi-Newton with adaptive step lengths for large-scale problems
• A fast quasi-Newton-type method for large-scale stochastic optimisation
• Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods
• Adaptive Sampling Quasi-Newton Methods for Derivative-Free Stochastic\n Optimization
• Exact and Inexact Subsampled Newton Methods for Optimization