A constrained optimization approach to improve robustness of neural networks

2024/09/18 by Shudian Zhao, Zhao, Shudian, Jan Kronqvist +1
Computer Science · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Neural Networks and Applications #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2409.13770

openalex publication_date 2024/09/18 · openalex created_date 2024/10/25 · openalex updated_date 2026/07/28

Abstract

In this paper, we present a novel nonlinear programming-based approach to fine-tune pre-trained neural networks to improve robustness against adversarial attacks while maintaining high accuracy on clean data. Our method introduces adversary-correction constraints to ensure correct classification of adversarial data and minimizes changes to the model parameters. We propose an efficient cutting-plane-based algorithm to iteratively solve the large-scale nonconvex optimization problem by approximating the feasible region through polyhedral cuts and balancing between robustness and accuracy. Computational experiments on standard datasets such as MNIST and CIFAR10 demonstrate that the proposed approach significantly improves robustness, even with a very small set of adversarial data, while maintaining minimal impact on accuracy.

Related

• Towards Deep Learning Models Resistant to Adversarial Attacks
• Revisiting Min-Max Optimization Problem in Adversarial Training
• Fast Training of Deep Neural Networks Robust to Adversarial Perturbations
• On Certifying Non-uniform Bound against Adversarial Attacks
• Bridging the Gap Between Adversarial Robustness and Optimization Bias