Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers

2020/02/22 by Chen Zhu, Renkun Ni, Zhu, Chen +9
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2002.09766

openalex publication_date 2020/02/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical robustness. In principle, convex relaxation can provide tight bounds if the solution to the relaxed problem is feasible for the original non-convex problem. We propose two regularizers that can be used to train neural networks that yield tighter convex relaxation bounds for robustness. In all of our experiments, the proposed regularizers result in higher certified accuracy than non-regularized baselines.

Citations

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• An approach to reachability analysis for feed-forward ReLU neural networks
• A Dual Approach to Scalable Verification of Deep Networks
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Provably Minimally-Distorted Adversarial Examples
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models

Related

• Tightened Convex Relaxations for Neural Network Robustness Certification
• A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
• Semidefinite relaxations for certifying robustness to adversarial\n examples
• On the tightness of linear relaxation based robustness certification methods
• The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification