SHIELD: Secure Hypernetworks for Incremental Expansion Learning Defense

2025/06/09 by Patryk Krukowski, Krukowski, Patryk, Łukasz Gorczyca +7
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2506.08255

openalex publication_date 2025/06/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Continual learning under adversarial conditions remains an open problem, as existing methods often compromise either robustness, scalability, or both. We propose a novel framework that integrates Interval Bound Propagation (IBP) with a hypernetwork-based architecture to enable certifiably robust continual learning across sequential tasks. Our method, SHIELD, generates task-specific model parameters via a shared hypernetwork conditioned solely on compact task embeddings, eliminating the need for replay buffers or full model copies and enabling efficient over time. To further enhance robustness, we introduce Interval MixUp, a novel training strategy that blends virtual examples represented as ℓ_∞ balls centered around MixUp points. Leveraging interval arithmetic, this technique guarantees certified robustness while mitigating the wrapping effect, resulting in smoother decision boundaries. We evaluate SHIELD under strong white-box adversarial attacks, including PGD and AutoAttack, across multiple benchmarks. It consistently outperforms existing robust continual learning methods, achieving state-of-the-art average accuracy while maintaining both scalability and certification. These results represent a significant step toward practical and theoretically grounded continual learning in adversarial settings.

Citations

• Overcoming catastrophic forgetting in neural networks
• Defense without Forgetting: Continual Adversarial Defense with Anisotropic & Isotropic Pseudo Replay
• Maintaining Adversarial Robustness in Continuous Learning
• HyperMask: Adaptive Hypernetwork-based Masks for Continual Learning
• A Comprehensive Survey of Continual Learning: Theory, Method and Application
• Susceptibility of Continual Learning Against Adversarial Attacks
• Continual Learning with Recursive Gradient Optimization
• Reliable Adversarial Distillation with Unreliable Teachers
• Posterior Meta-Replay for Continual Learning
• Bag of Tricks for Adversarial Training
• Supermasks in Superposition
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Continual learning with hypernetworks
• Fast and Stable Interval Bounds Propagation for Training Verifiably Robust Models
• Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
• Efficient Neural Network Robustness Certification with General Activation Functions
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• Memory Efficient Experience Replay for Streaming Learning
• DARTS: Deceiving Autonomous Cars with Toxic Signs
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models
• Training Ensembles to Detect Adversarial Examples
• Memory Aware Synapses: Learning what (not) to forget
• PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
• Mitigating Adversarial Effects Through Randomization
• mixup: Beyond Empirical Risk Minimization
• One Pixel Attack for Fooling Deep Neural Networks
• APE-GAN: Adversarial Perturbation Elimination with GAN
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Gradient Episodic Memory for Continual Learning
• CORe50: a New Dataset and Benchmark for Continuous Object Recognition
• HyperNetworks
• Less-forgetting Learning in Deep Neural Networks
• Learning without Forgetting
• Progressive Neural Networks
• Wide Residual Networks
• Deep Residual Learning for Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks

Related

• Fast and Stable Interval Bounds Propagation for Training Verifiably Robust Models
• Towards Adversarially Robust Continual Learning
• Enhancing Accuracy and Robustness through Adversarial Training in Class Incremental Continual Learning
• Parameter-Saving Adversarial Training: Reinforcing Multi-Perturbation Robustness via Hypernetworks
• Susceptibility of Continual Learning Against Adversarial Attacks