Preconditioning Natural and Second Order Gradient Descent in Quantum Optimization: A Performance Benchmark

2025/04/23 by Théo Lisart-Liebermann, Lisart-Liebermann, Théo, Arcesio Castañeda Medina +1 · 1 voice
Biochemistry, Genetics and Molecular Biology · Computer Science · Physics and Astronomy · #Benchmark (surveying) #Broyden–Fletcher–Goldfarb–Shanno algorithm #Computational Engineering #Convergence (economics) #FOS: Computer and information sciences #FOS: Physical sciences #Finance #Gradient descent #Parametric statistics #Quantum Computing Algorithms and Architecture #Quantum Information and Cryptography #Quantum Physics (quant-ph) #Selection (genetic algorithm) #Sensitivity (control systems) #Spectroscopy Techniques in Biomedical and Chemical Research #Stochastic gradient descent #and Science (cs.CE) #cs.CE #quant-ph

paper · pdf · doi:10.48550/arxiv.2504.16518

openalex publication_date 2025/04/23 · arxiv published 2025/04/23 · arxiv updated 2025/04/23 · openalex created_date 2025/10/11 · openalex updated_date 2026/08/05

Abstract

The optimization of parametric quantum circuits is technically hindered by three major obstacles: the non-convex nature of the objective function, noisy gradient evaluations, and the presence of barren plateaus. As a result, the selection of classical optimizer becomes a critical factor in assessing and exploiting quantum-classical applications. One promising approach to tackle these challenges involves incorporating curvature information into the parameter update. The most prominent methods in this field are quasi-Newton and quantum natural gradient methods, which can facilitate faster convergence compared to first-order approaches. Second order methods however exhibit a significant trade-off between computational cost and accuracy, as well as heightened sensitivity to noise. This study evaluates the performance of three families of optimizers on synthetically generated MaxCut problems on a shallow QAOA algorithm. To address noise sensitivity and iteration cost, we demonstrate that incorporating secant-penalization in the BFGS update rule (SP-BFGS) yields improved outcomes for QAOA optimization problems, introducing a novel approach to stabilizing BFGS updates against gradient noise.

Citations

• Noise-induced shallow circuits and absence of barren plateaus
• An Empirical Review of Optimization Techniques for Quantum Variational Circuits
• Fisher Information in Noisy Intermediate-Scale Quantum Applications
• Quantum variational optimization: The role of entanglement and problem hardness
• Cost Function Dependent Barren Plateaus in Shallow Parametrized Quantum Circuits
• Simultaneous Perturbation Stochastic Approximation of the Quantum Fisher Information
• Prospects of Quantum Computing for Molecular Sciences
• The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods
• Structure optimization for parameterized quantum circuits
• A Regularized Limited Memory BFGS method for Large-Scale Unconstrained Optimization and its Efficient Implementations
• Connecting ansatz expressibility to gradient magnitudes and barren plateaus
• Variational Quantum Algorithms
• Absence of Barren Plateaus in Quantum Convolutional Neural Networks
• The power of quantum neural networks
• Variational Quantum Algorithm for Estimating the Quantum Fisher Information
• From pulses to circuits and back again: A quantum optimal control perspective on variational quantum algorithms
• Estimating the gradient and higher-order derivatives on quantum hardware
• Characterizing the loss landscape of variational quantum circuits
• Quantum-optimal-control-inspired ansatz for variational quantum algorithms
• Using models to improve optimizers for variational quantum algorithms
• Avoiding local minima in variational quantum eigensolvers with the natural gradient optimizer
• Classical Optimizers for Noisy Intermediate-Scale Quantum Devices
• Hyper-Parameter Optimization: A Review of Algorithms and Applications
• Policy Gradient based Quantum Approximate Optimization Algorithm
• Gradient descent with momentum --- to accelerate or to super-accelerate?
• Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS
• Robust and efficient algorithms for high-dimensional black-box quantum optimization
• Quantum Natural Gradient
• Parameterized quantum circuits as machine learning models
• Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks
• Review of second-order optimization techniques in artificial neural networks backpropagation
• Quantum Generative Adversarial Networks for Learning and Loading Random Distributions
• Evaluating analytic gradients on quantum hardware
• Quantum computational chemistry
• Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration
• Performance of hybrid quantum/classical variational heuristics for combinatorial optimization
• Barren plateaus in quantum neural network training landscapes
• On the Convergence of Adam and Beyond
• Quantum Computing in the NISQ era and beyond
• Non-convex Optimization for Machine Learning
• A Robust Multi-Batch L-BFGS Method for Machine Learning
• Hardware-efficient Variational Quantum Eigensolver for Small Molecules and Quantum Magnets
• Strategies for quantum computing molecular energies using the unitary coupled cluster ansatz
• Quantum sensing
• Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
• Quantum Supremacy through the Quantum Approximate Optimization Algorithm
• A survey of dimensionality reduction techniques
• Adaptive hybrid optimal quantum control for imprecisely characterized systems
• Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods
• Quantum Simulation
• Generating Sequences With Recurrent Neural Networks
• A variational eigenvalue solver on a photonic quantum processor
• Practical Bayesian Optimization of Machine Learning Algorithms
• Portfolio Allocation for Bayesian Optimization
• Quantum algorithm for solving linear systems of equations
• Quantum Computation by Adiabatic Evolution
• Simulating physics with computers
• Natural Gradient Works Efficiently in Learning
• Conditioning of quasi-Newton methods for function minimization
• A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise
• Taking the Human Out of the Loop: A Review of Bayesian Optimization
• A Simplex Method for Function Minimization
• Some methods of speeding up the convergence of iteration methods

Discussions

• This is our first paper in a series of two, which tackles the question of trainability of shallow QAOA algorithms. We provide benchmarks for quasi-newton and natural gradient methods, in addition to i [bsky, 1 points, 1 comments]

Related

• Quantum Natural Gradient optimizer on noisy platforms: QAOA as a case study
• LAWS: Look Around and Warm-Start Natural Gradient Descent for Quantum Neural Networks
• Benchmarking Lie-Algebraic Pretraining and Non-Variational QWOA for the MaxCut Problem
• Quantum Optimization via Gradient-Based Hamiltonian Descent
• Fast Convex Optimization with Quantum Gradient Methods