Generalization ability and Vulnerabilities to adversarial perturbations: Two sides of the same coin

2022/05/22 by Jung Hoon Lee, Lee, Jung Hoon, Sujith Vijayan +1
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Integrated Circuits and Semiconductor Failure Analysis #Machine Learning (cs.LG) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2205.10952

openalex publication_date 2022/05/22 · openalex created_date 2022/05/26 · openalex updated_date 2026/07/28

Abstract

Deep neural networks (DNNs), the agents of deep learning (DL), require a massive number of parallel/sequential operations, which makes it difficult to comprehend them and impedes proper diagnosis. Without better knowledge of DNNs' internal process, deploying DNNs in high-stakes domains may lead to catastrophic failures. Therefore, to build more reliable DNNs/DL, it is imperative that we gain insights into their underlying decision-making process. Here, we use the self-organizing map (SOM) to analyze DL models' internal codes associated with DNNs' decision-making. Our analyses suggest that shallow layers close to the input layer map onto homogeneous codes and that deep layers close to the output layer transform these homogeneous codes in shallow layers to diverse codes. We also found evidence indicating that homogeneous codes may underlie DNNs' vulnerabilities to adversarial perturbations.

Related

• Meet You Halfway: Explaining Deep Learning Mysteries
• Understanding Adversarial Behavior of DNNs by Disentangling Non-Robust and Robust Components in Performance Metric
• Library network, a possible path to explainable neural networks
• Understanding and Diagnosing Vulnerability under Adversarial Attacks
• Understanding Adversarial Examples Through Deep Neural Network's Response Surface and Uncertainty Regions