2025/02/05 by Marcus Carlsson, В. В. Никитин, Carlsson, Marcus +5 · 1 citation
Computer Science · Engineering · Mathematics · #FOS: Mathematics #Mathematical Approximation and Integration #Matrix Theory and Algorithms #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Spacecraft Dynamics and Control
paper · pdf · doi:10.48550/arxiv.2502.03070
openalex publication_date 2025/02/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
Second order information is useful in many ways in smooth optimization problems, including for the design of step size rules and descent directions, or the analysis of the local properties of the objective functional. However, the computation and storage of the Hessian matrix using second order partial derivatives is prohibitive in many contexts, and in particular in large scale problems. In this work, we propose a new framework for computing and presenting second order information in analytic form. The key novel insight is that the Hessian for a problem can be worked with efficiently by computing its bilinear form or operator form using Taylor expansions, instead of introducing a basis and then computing the Hessian matrix. Our new framework is suited for high-dimensional problems stemming e.g. from imaging applications, where computation of the Hessian matrix is unfeasible. We also show how this can be used to implement Newton's step rule, Daniel's Conjugate Gradient rule, or Quasi-Newton schemes, without explicit knowledge of the Hessian matrix, and illustrate our findings with a simple numerical experiment.