Accelerating Quadratic Optimization with Reinforcement Learning

2021/07/22 by Jeffrey Ichnowski, Ichnowski, Jeffrey, Paras Jain +15 · 11 citations
Computer Science · Decision Sciences · Mathematics · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning and Data Classification #Optimization and Control (math.OC) #Reinforcement Learning in Robotics #cs.LG #math.OC

paper · pdf · doi:10.48550/arxiv.2107.10847

25 pages, 7 figures. Code available at https://github.com/berkeleyautomation/rlqp

arxiv created 2021/07/22 · openalex publication_date 2021/07/22 · arxiv updated 2021/07/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

First-order methods for quadratic optimization such as OSQP are widely used for large-scale machine learning and embedded optimal control, where many related problems must be rapidly solved. These methods face two persistent challenges: manual hyperparameter tuning and convergence time to high-accuracy solutions. To address these, we explore how Reinforcement Learning (RL) can learn a policy to tune parameters to accelerate convergence. In experiments with well-known QP benchmarks we find that our RL policy, RLQP, significantly outperforms state-of-the-art QP solvers by up to 3x. RLQP generalizes surprisingly well to previously unseen problems with varying dimension and structure from different applications, including the QPLIB, Netlib LP and Maros-Meszaros problems. Code for RLQP is available at https://github.com/berkeleyautomation/rlqp.

Citations

• Benchmarking optimization software with performance profiles
• Continuous control with deep reinforcement learning
• Learning to Optimize
• A Note on Performance Profiles for Benchmarking Software
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Learning Combinatorial Optimization Algorithms over Graphs
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• OSQP: An Operator Splitting Solver for Quadratic Programs
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• On First-Order Meta-Learning Algorithms
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
• Machine learning for combinatorial optimization: A methodological tour d’horizon

Cited by

• Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding
• Warm-starting active-set solvers using graph neural networks
• Data-driven Projection Generation for Efficiently Solving Heterogeneous Quadratic Programming Problems
• On the Relationships among GPU-Accelerated First-Order Methods for Solving Linear Programming
• Reinforcement learning for adaptive interior point methods in convex quadratic programming
• Learning to accelerate distributed ADMM using graph neural networks
• Solving Quadratic Programs via Deep Unrolled Douglas-Rachford Splitting
• Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms
• Learning Acceleration Algorithms for Fast Parametric Convex Optimization with Certified Robustness
• Robust and Efficient Embedded Convex Optimization through First-Order Adaptive Caching
• A Learning-Based Inexact ADMM for Solving Quadratic Programs

Related

• Reinforcement learning for adaptive interior point methods in convex quadratic programming
• NeuralQP: A General Hypergraph-based Optimization Framework for Large-scale QCQPs
• Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding
• Using Deep Q-Learning to Control Optimization Hyperparameters
• Data-Driven LQR using Reinforcement Learning and Quadratic Neural Networks