Reinforced model predictive control (RL-MPC) for building energy management

2022/01/07 by Javier Arroyo, Carlo Manna, Fred Spiessens +1 · 6 citations
Engineering · #Advanced Control Systems Optimization #Building Energy and Comfort Optimization #Smart Grid Energy Management

paper · pdf · doi:10.1016/j.apenergy.2021.118346

openalex publication_date 2022/01/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/03

Abstract

Buildings need advanced control for the efficient and climate-neutral use of their energy systems. Model predictive control (MPC) and reinforcement learning (RL) arise as two powerful control techniques that have been extensively investigated in the literature for their application to building energy management. These methods show complementary qualities in terms of constraint satisfaction, computational demand, adaptability, and intelligibility, but usually a choice is made between both approaches. This paper compares both control approaches and proposes a novel algorithm called reinforced predictive control (RL-MPC) that merges their relative merits. First, the complementarity between RL and MPC is emphasized on a conceptual level by commenting on the main aspects of each method. Second, the RL-MPC algorithm is described that effectively combines features from each approach, namely state estimation, dynamic optimization, and learning. Finally, MPC, RL, and RL-MPC are implemented and evaluated in BOPTEST, a standardized simulation framework for the assessment of advanced control algorithms in buildings. The results indicate that pure RL cannot provide constraint satisfaction when using a control formulation equivalent to MPC and the same controller model for learning. The new RL-MPC algorithm can meet constraints and provide similar performance to MPC while enabling continuous learning and the possibility to deal with uncertain environments.

Citations

• Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control
• Transcription Methods for Trajectory Optimization: a beginners tutorial
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learning
• Deep Reinforcement Learning for Optimal Control of Space Heating
• Differentiable MPC for End-to-end Planning and Control
• Information Theoretic Model Predictive Q-Learning

Cited by

• On Architectures for Combining Reinforcement Learning and Model Predictive Control with Runtime Improvements
• Model Predictive Control-Guided Reinforcement Learning for Implicit Balancing
• A learning model predictive control for virtual coupling in railroads
• Probabilistic Forecasting for Building Energy Systems using Time-Series Foundation Models
• An Online Machine Learning Multi-resolution Optimization Framework for Energy System Design Limit of Performance Analysis
• Hybrid Reinforcement Learning and Model Predictive Control for Adaptive Control of Hydrogen-Diesel Dual-Fuel Combustion

Related

• Data-driven Model Predictive and Reinforcement Learning Based Control for Building Energy Management: a Survey
• Lessons learned from field demonstrations of model predictive control and reinforcement learning for residential and commercial HVAC: A review
• Generalized Reinforcement Learning for Building Control using Behavioral Cloning
• PhysQ: A Physics Informed Reinforcement Learning Framework for Building Control
• Robust Learning Model Predictive Control for Periodically Correlated Building Control