Robust Deep Neural Networks Inspired by Fuzzy Logic

2019/11/20 by Minh Le, Minh Nghia Le, Le, Minh · 1 voice
Computer Science · #cs.LG #cs.LO #stat.ML

paper · pdf · doi:10.48550/arxiv.1911.08635

Abstract

Deep neural networks have achieved impressive performance and become the de-facto standard in many tasks. However, troubling phenomena such as adversarial and fooling examples suggest that the generalization they make is flawed. I argue that among the roots of the phenomena are two geometric properties of common deep learning architectures: their distributed nature and the connectedness of their decision regions. As a remedy, I propose new architectures inspired by fuzzy logic that combine several alternative design elements. Through experiments on MNIST and CIFAR-10, the new models are shown to be more local, better at rejecting noise samples, and more robust against adversarial examples. Ablation analyses reveal behaviors on adversarial examples that cannot be explained by the linearity hypothesis but are consistent with the hypothesis that logic-inspired traits create more robust models.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Gradient-based learning applied to document recognition

Discussions

• Logic-Inspired Deep Neural Networks [hn, 4 points, 0 comments]

Related

• Optimism in the Face of Adversity: Understanding and Improving Deep Learning through Adversarial Robustness
• Intriguing Properties of Adversarial Examples
• Out-distribution training confers robustness to deep neural networks
• A Theoretical Framework for Robustness of (Deep) Classifiers against Adversarial Examples
• Less is More: Culling the Training Set to Improve Robustness of Deep Neural Networks