Risk Bounds for Robust Deep Learning

2020/09/14 by Johannes Lederer, Lederer, Johannes · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Neural and Evolutionary Computing (cs.NE) #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2009.06202

openalex publication_date 2020/09/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

It has been observed that certain loss functions can render deep-learning pipelines robust against flaws in the data. In this paper, we support these empirical findings with statistical theory. We especially show that empirical-risk minimization with unbounded, Lipschitz-continuous loss functions, such as the least-absolute deviation loss, Huber loss, Cauchy loss, and Tukey's biweight loss, can provide efficient prediction under minimal assumptions on the data. More generally speaking, our paper provides theoretical evidence for the benefits of robust loss functions in deep learning.

Citations

• Studying Very Low Resolution Recognition Using Deep Networks
• Adversarial Machine Learning at Scale
• Ensemble Adversarial Training: Attacks and Defenses
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Towards Robust Deep Neural Networks

Cited by

• Optimization Landscapes of Wide Deep Neural Networks Are Benign

Related

• Learning without Concentration for General Loss Functions
• Robust deep learning from weakly dependent data
• Transgressing the boundaries: towards a rigorous understanding of deep learning and its (non-)robustness
• Theory II: Landscape of the Empirical Risk in Deep Learning
• A straightforward line search approach on the expected empirical loss for stochastic deep learning problems