End-to-End Reinforcement Learning of Koopman Models for eNMPC of an Air Separation Unit

2025/11/06 by Daniel Mayfrank, Kayra Dernek, Mayfrank, Daniel +7
Engineering · Physics and Astronomy · #Advanced Control Systems Optimization #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Optimization and Control (math.OC) #Process Optimization and Integration

paper · doi:10.48550/arxiv.2511.04522

openalex publication_date 2025/11/06 · openalex created_date 2025/11/08 · openalex updated_date 2026/07/28

Abstract

With our recently proposed method based on reinforcement learning (Mayfrank et al. (2024), Comput. Chem. Eng. 190), Koopman surrogate models can be trained for optimal performance in specific (economic) nonlinear model predictive control ((e)NMPC) applications. So far, our method has exclusively been demonstrated on a small-scale case study. Herein, we show that our method scales well to a more challenging demand response case study built on a large-scale model of a single-product (nitrogen) air separation unit. Across all numerical experiments, we assume observability of only a few realistically measurable plant variables. Compared to a purely system identification-based Koopman eNMPC, which generates small economic savings but frequently violates constraints, our method delivers similar economic performance while avoiding constraint violations.

Citations

• Data-driven End-to-end Learning of Pole Placement Control for Nonlinear Dynamics via Koopman Invariant Subspaces
• Differentiable Convex Optimization Layers
• Differentiable MPC for End-to-end Planning and Control
• Proximal Policy Optimization Algorithms
• Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control
• A practical scenario generation method for electricity prices on day-ahead and intraday spot markets

Related

• Sample-Efficient Reinforcement Learning of Koopman eNMPC
• Task-optimal data-driven surrogate models for eNMPC via differentiable simulation and optimization
• Leveraging Reinforcement Learning and Koopman Theory for Enhanced Model Predictive Control Performance
• Deep Koopman Economic Model Predictive Control of a Pasteurisation Unit
• Machine learning-based input-augmented Koopman modeling and predictive control of nonlinear processes