Evaluating the Adversarial Robustness of Adaptive Test-time Defenses

2022/02/28 by Francesco Croce, Croce, Francesco, Sven Gowal +8 · 3 citations
Computer Science · Medicine · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Medical Imaging Techniques and Applications

paper · pdf · doi:10.48550/arxiv.2202.13711

openalex publication_date 2022/02/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Adaptive defenses, which optimize at test time, promise to improve adversarial robustness. We categorize such adaptive test-time defenses, explain their potential benefits and drawbacks, and evaluate a representative variety of the latest adaptive defenses for image classification. Unfortunately, none significantly improve upon static defenses when subjected to our careful case study evaluation. Some even weaken the underlying static model while simultaneously increasing inference computation. While these results are disappointing, we still believe that adaptive test-time defenses are a promising avenue of research and, as such, we provide recommendations for their thorough evaluation. We extend the checklist of Carlini et al. (2019) by providing concrete steps specific to adaptive defenses.

Cited by

• FedPoisonTTP: A Threat Model and Poisoning Attack for Federated Test-Time Personalization
• Enhancing CLIP Robustness via Cross-Modality Alignment
• ADVMEM: Adversarial Memory Initialization for Realistic Test-Time Adaptation via Tracklet-Based Benchmarking

Related

• An Adaptive View of Adversarial Robustness from Test-time Smoothing Defense
• The Efficacy of SHIELD under Different Threat Models
• Robust Feature Inference: A Test-time Defense Strategy using Spectral Projections
• Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attacks
• Leveraging Generalizability of Image-to-Image Translation for Enhanced Adversarial Defense