In Proximity of ReLU DNN, PWA Function, and Explicit MPC

2020/06/09 by Saman Fahandezh-Saadi, Masayoshi Tomizuka, Fahandezh-Saadi, Saman +1
Computer Science · Engineering · Mathematics · #Advanced Control Systems Optimization #Control Systems and Identification #FOS: Computer and information sciences #FOS: Electrical engineering #Fault Detection and Control Systems #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Systems and Control (eess.SY) #cs.LG #cs.SY #eess.SY #electronic engineering #information engineering #stat.ML

paper · pdf · doi:10.48550/arxiv.2006.05001

Submitted to Conference on Decision and Control (CDC) 2020

openalex publication_date 2020/06/09 · arxiv created 2020/11/05 · arxiv updated 2020/11/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Rectifier (ReLU) deep neural networks (DNN) and their connection with piecewise affine (PWA) functions is analyzed. The paper is an effort to find and study the possibility of representing explicit state feedback policy of model predictive control (MPC) as a ReLU DNN, and vice versa. The complexity and architecture of DNN has been examined through some theorems and discussions. An approximate method has been developed for identification of input-space in ReLU net which results a PWA function over polyhedral regions. Also, inverse multiparametric linear or quadratic programs (mp-LP or mp-QP) has been studied which deals with reconstruction of constraints and cost function given a PWA function.

Citations

• Deep Knowledge Tracing
• On the number of response regions of deep feed forward networks with piece-wise linear activations
• ImageNet classification with deep convolutional neural networks
• Safe Model-based Reinforcement Learning with Stability Guarantees

Related

• Dissecting Deep Neural Networks
• Assured Neural Network Architectures for Control and Identification of Nonlinear Systems
• Theoretical Exploration of Solutions of Feedforward ReLU Networks
• YANNs: Y-wise Affine Neural Networks for Exact and Efficient Representations of Piecewise Linear Functions
• Deep Representation with ReLU Neural Networks