2018/07/02 by André Gustavo Carlon, Ben Mansour Dia, Carlon, Andre Gustavo +7 · 1 citation
Computer Science · Decision Sciences · #62K05 #65C05 #65C60 #65N21 #Advanced Multi-Objective Optimization Algorithms #FOS: Mathematics #Numerical Analysis (math.NA) #Probabilistic and Robust Engineering Design #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1807.00653
openalex publication_date 2018/07/02 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28
Finding the best setup for experiments is the primary concern for Optimal\nExperimental Design (OED). Here, we focus on the Bayesian experimental design\nproblem of finding the setup that maximizes the Shannon expected information\ngain. We use the stochastic gradient descent and its accelerated counterpart,\nwhich employs Nesterov's method, to solve the optimization problem in OED. We\nadapt a restart technique, originally proposed for the acceleration in\ndeterministic optimization, to improve stochastic optimization methods. We\ncombine these optimization methods with three estimators of the objective\nfunction: the double-loop Monte Carlo estimator (DLMC), the Monte Carlo\nestimator using the Laplace approximation for the posterior distribution (MCLA)\nand the double-loop Monte Carlo estimator with Laplace-based importance\nsampling (DLMCIS). Using stochastic gradient methods and Laplace-based\nestimators together allows us to use expensive and complex models, such as\nthose that require solving partial differential equations (PDEs). From a\ntheoretical viewpoint, we derive an explicit formula to compute the gradient\nestimator of the Monte Carlo methods, including MCLA and DLMCIS. From a\ncomputational standpoint, we study four examples: three based on analytical\nfunctions and one using the finite element method. The last example is an\nelectrical impedance tomography experiment based on the complete electrode\nmodel. In these examples, the accelerated stochastic gradient descent method\nusing MCLA converges to local maxima with up to five orders of magnitude fewer\nmodel evaluations than gradient descent with DLMC.\n