Successive convex optimization for transformer encoder model predictive control

2026/05/31 by Xingxiao Chen, Mark Cannon
Mathematics · #math.OC

paper · pdf

arxiv created 2026/07/30 · arxiv updated 2026/07/31

Abstract

We propose a data-driven Model Predictive Control (MPC) framework that employs a transformer encoder to generate multi-step predictions. To handle the nonconvex attention mechanism, we derive difference of convex (DC) representations of the transformer encoder components and embed them in a successive convex programming (SCP) iteration. Recursive feasibility and convergence of the SCP iterates are guaranteed, and each iterate yields a solution estimate satisfying the problem constraints. Under mild assumptions, the SCP iteration converges to a locally optimal solution of the MPC problem. The approach is illustrated on a benchmark nonlinear control problem.

Citations

• Deep learning adaptive Model Predictive Control of Fed-Batch Cultivations
• Convexifying Transformers: Improving optimization and understanding of transformer networks
• Recurrent Neural Network-based Model Predictive Control for Continuous Pharmaceutical Manufacturing
• Attention Is All You Need
• Deep Residual Learning for Image Recognition

Related

• TransformerMPC: Accelerating Model Predictive Control via Transformers
• Transformer-Based Model Predictive Path Integral Control
• TransMPC: Transformer-based Explicit MPC with Variable Prediction Horizon
• Learning a convex cost-to-go for single step model predictive control
• A Convex Feasibility Approach to Anytime Model Predictive Control