Building Compact and Robust Deep Neural Networks with Toeplitz Matrices

2021/09/02 by Alexandre Araujo, Araujo, Alexandre
Computer Science · #Adversarial Robustness in Machine Learning #Advanced Neural Network Applications #Anomaly Detection Techniques and Applications

paper · pdf · doi:10.48550/arxiv.2109.00959

Abstract

Deep neural networks are state-of-the-art in a wide variety of tasks, however, they exhibit important limitations which hinder their use and deployment in real-world applications. When developing and training neural networks, the accuracy should not be the only concern, neural networks must also be cost-effective and reliable. Although accurate, large neural networks often lack these properties. This thesis focuses on the problem of training neural networks which are not only accurate but also compact, easy to train, reliable and robust to adversarial examples. To tackle these problems, we leverage the properties of structured matrices from the Toeplitz family to build compact and secure neural networks.

Citations

• Some Mathematical Notes on Three-Mode Factor Analysis
• The perceptron: A probabilistic model for information storage and organization in the brain.
• An algorithm for the machine calculation of complex Fourier series
• Gradient-based learning applied to document recognition
• ImageNet classification with deep convolutional neural networks

Related

• Differentiable Learning of Generalized Structured Matrices for Efficient Deep Neural Networks
• LU decomposition and Toeplitz decomposition of a neural network
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Refining the Structure of Neural Networks Using Matrix Conditioning