Compositional Curvature Bounds for Deep Neural Networks

2024/06/07 by Taha Entesari, Entesari, Taha, Sina Sharifi +3 · 1 citation
Engineering · #Advanced X-ray and CT Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Medical Imaging and Analysis

paper · pdf · doi:10.48550/arxiv.2406.05119

openalex publication_date 2024/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A key challenge that threatens the widespread use of neural networks in safety-critical applications is their vulnerability to adversarial attacks. In this paper, we study the second-order behavior of continuously differentiable deep neural networks, focusing on robustness against adversarial perturbations. First, we provide a theoretical analysis of robustness and attack certificates for deep classifiers by leveraging local gradients and upper bounds on the second derivative (curvature constant). Next, we introduce a novel algorithm to analytically compute provable upper bounds on the second derivative of neural networks. This algorithm leverages the compositional structure of the model to propagate the curvature bound layer-by-layer, giving rise to a scalable and modular approach. The proposed bound can serve as a differentiable regularizer to control the curvature of neural networks during training, thereby enhancing robustness. Finally, we demonstrate the efficacy of our method on classification tasks using the MNIST and CIFAR-10 datasets.

Cited by

• Curvature Dynamic Black-box Attack: revisiting adversarial robustness via dynamic curvature estimation

Related

• Efficiently Training Low-Curvature Neural Networks
• Second Order Optimization for Adversarial Robustness and Interpretability
• Biologically inspired protection of deep networks from adversarial attacks
• Improving Resistance to Adversarial Deformations by Regularizing Gradients
• On Connections between Regularizations for Improving DNN Robustness