Analysis of Invariance and Robustness via Invertibility of ReLU-Networks

2018/06/25 by Jens Behrmann, Sören Dittmer, Behrmann, Jens +5
Computer Science · Engineering · Physics and Astronomy · #Advancements in Semiconductor Devices and Circuit Design #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks

paper · pdf · doi:10.48550/arxiv.1806.09730

openalex publication_date 2018/06/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Studying the invertibility of deep neural networks (DNNs) provides a principled approach to better understand the behavior of these powerful models. Despite being a promising diagnostic tool, a consistent theory on their invertibility is still lacking. We derive a theoretically motivated approach to explore the preimages of ReLU-layers and mechanisms affecting the stability of the inverse. Using the developed theory, we numerically show how this approach uncovers characteristic properties of the network.

Citations

• The Loss Surfaces of Multilayer Networks
• Understanding deep convolutional networks
• Parseval Networks: Improving Robustness to Adversarial Examples
• Methods for Interpreting and Understanding Deep Neural Networks
• Mathematics of Deep Learning
• Learning Multiple Layers of Features from Tiny Images

Related

• Reverse-Engineering Deep ReLU Networks
• Globally Injective ReLU Networks
• Injectivity of ReLU-layers: Tools from Frame Theory
• Finding Input Characterizations for Output Properties in ReLU Neural Networks
• Complexity of Injectivity and Verification of ReLU Neural Networks