Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

2019/03/28 by Dan Hendrycks, Thomas Dietterich, Thomas G. Dietterich +2 · 390 citations
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.CV #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1903.12261

ICLR 2019 camera-ready; datasets available at https://github.com/hendrycks/robustness ; this article supersedes arXiv:1807.01697

arxiv created 2019/03/28 · openalex publication_date 2019/03/28 · arxiv updated 2019/04/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations. Unlike recent robustness research, this benchmark evaluates performance on common corruptions and perturbations not worst-case adversarial perturbations. We find that there are negligible changes in relative corruption robustness from AlexNet classifiers to ResNet classifiers. Afterward we discover ways to enhance corruption and perturbation robustness. We even find that a bypassed adversarial defense provides substantial common perturbation robustness. Together our benchmarks may aid future work toward networks that robustly generalize.

Citations

• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations

Cited by

• VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models
• MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents
• AGVBench: A Reliability-Oriented Benchmark of Data Augmentation for Vein Recognition
• Controllable Diversity in Normalization-Based Implicit Ensembles via Softmax-Temperature Modulation
• Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations
• Source-Free Controlled Adaptation of Teachers for Continual Test-Time Adaptation
• Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation
• Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation
• PIcsC: Partitioning-Induced Covariate Shift Correction
• ReLATE: Reliability-Guided Evidence Fusion for Robust UAV--Satellite cross-view Geo-Localization
• Structural Preservation Governs Data Augmentation in Deep Learning-Based Laser Speckle Material Classification
• Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation
• Granular Ball Guided Masking: Structure-aware Data Augmentation
• Clever Hans in Chemistry: Chemist Style Signals Confound Activity Prediction on Public Benchmarks
• Probing and Leveraging Video Diffusion Transformer Features for Robust Point Tracking
• Multi-Layer Confidence Scoring for Detection of Out-of-Distribution Samples, Adversarial Attacks, and In-Distribution Misclassifications
• Efficient Jailbreak Mitigation Using Semantic Linear Classification in a Multi-Staged Pipeline
• Can We Test Consciousness Theories on AI? Ablations, Markers, and Robustness
• The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs
• Stylized Synthetic Augmentation further improves Corruption Robustness
• SUPER -- A Framework for Sensitivity-based Uncertainty-aware Performance and Risk Assessment in Visual Inertial Odometry
• From Code to Field: Evaluating the Robustness of Convolutional Neural Networks for Disease Diagnosis in Mango Leaves
• Efficient Vision-Language Reasoning via Adaptive Token Pruning
• Progressive Conditioned Scale-Shift Recalibration of Self-Attention for Online Test-time Adaptation
• Audio-Visual Camera Pose Estimation with Passive Scene Sounds and In-the-Wild Video
• On the Bayes Inconsistency of Disagreement Discrepancy Surrogates
• Neural Collapse in Test-Time Adaptation
• Hands-on Evaluation of Visual Transformers for Object Recognition and Detection
• Uncertainty-Aware Subset Selection for Robust Visual Explainability under Distribution Shifts
• Instance-Aware Test-Time Segmentation for Continual Domain Shifts
• Towards Reliable Test-Time Adaptation: Style Invariance as a Correctness Likelihood
• Intra-Class Probabilistic Embeddings for Uncertainty Estimation in Vision-Language Models
• Exploiting Domain Properties in Language-Driven Domain Generalization for Semantic Segmentation
• Instant Video Models: Universal Adapters for Stabilizing Image-Based Networks
• Walking on the Fiber: A Simple Geometric Approximation for Bayesian Neural Networks
• RoboDriveVLM: A Novel Benchmark and Baseline towards Robust Vision-Language Models for Autonomous Driving
• Adam Simplified: Bias Correction Debunked
• Enhancing Conformal Prediction via Class Similarity
• Benchmarking Corruption Robustness of LVLMs: A Discriminative Benchmark and Robustness Alignment Metric
• DriveFlow: Rectified Flow Adaptation for Robust 3D Object Detection in Autonomous Driving
• LookSharp: Attention Entropy Minimization for Test-Time Adaptation
• PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property Estimation
• DHAuDS: A Dynamic and Heterogeneous Audio Benchmark for Test-Time Adaptation
• Concept Regions Matter: Benchmarking CLIP with a New Cluster-Importance Approach
• Understanding Private Learning From Feature Perspective
• Vision Language Models are Confused Tourists
• ILoRA: Federated Learning with Low-Rank Adaptation for Heterogeneous Client Aggregation
• A Dataset and Baseline for Deep Learning-Based Visual Quality Inspection in Remanufacturing
• SNAP: Low-Latency Test-Time Adaptation with Sparse Updates
• Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machines
• Data Value in the Age of Scaling: Understanding LLM Scaling Dynamics Under Real-Synthetic Data Mixtures
• RobustGait: Robustness Analysis for Appearance Based Gait Recognition
• A Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning
• DINO-Detect: A Simple yet Effective Framework for Blur-Robust AI-Generated Image Detection
• Uncover and Unlearn Nuisances: Agnostic Fully Test-Time Adaptation
• Stabilizing Multi-Attack Adversarial Training via Bandit Optimization
• Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification
• Test-Time Spectrum-Aware Latent Steering for Zero-Shot Generalization in Vision-Language Models
• "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models
• Provable Repair of Deep Neural Network Defects by Preimage Synthesis and Property Refinement
• ClusterMine: Robust Label-Free Visual Out-Of-Distribution Detection via Concept Mining from Text Corpora
• RobustA: Robust Anomaly Detection in Multimodal Data
• DARN: Dynamic Adaptive Regularization Networks for Efficient and Robust Foundation Model Adaptation
• Decoupled Entropy Minimization
• Test Time Adaptation Using Adaptive Quantile Recalibration
• Learning with Category-Equivariant Architectures for Human Activity Recognition
• Purrturbed but Stable: Human-Cat Invariant Representations Across CNNs, ViTs and Self-Supervised ViTs
• Bayesian Natural Gradient Fine-Tuning of CLIP Models via Kalman Filtering
• T3: Test-Time Model Merging in VLMs for Zero-Shot Medical Imaging Analysis
• ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
• C-LEAD: Contrastive Learning for Enhanced Adversarial Defense
• VISAT: Benchmarking Adversarial and Distribution Shift Robustness in Traffic Sign Recognition with Visual Attributes
• Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Tasks
• EgoSafe: A First-Person Mobile-Captured Benchmark for Visual Safety Understanding
• MAPS: A Synthetic Dataset for Probing Vision Models in a Controlled 3D Scene Space
• Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
• Humanoid-inspired Causal Representation Learning for Domain Generalization
• Eigen-Value: Efficient Domain-Robust Data Valuation via Eigenvalue-Based Approach
• The Benchmarking Epistemology: Construct Validity for Evaluating Machine Learning Models
• Prediction-Powered Semi-Supervised Learning with Online Power Tuning
• Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptions
• Buffer layers for Test-Time Adaptation
• H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition
• What Does It Take to Build a Performant Selective Classifier?
• CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks
• Knowledge Distillation of Uncertainty using Deep Latent Factor Model
• Uncertainty Estimation by Flexible Evidential Deep Learning
• Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformers
• Domain Generalizable Continual Learning
• Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness
• SteeringTTA: Guiding Diffusion Trajectories for Robust Test-Time-Adaptation
• Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Training
• A Function-Centric Perspective on Flat and Sharp Minima
• Class-aware Domain Knowledge Fusion and Fission for Continual Test-Time Adaptation
• Adversarial Attacks Leverage Interference Between Features in Superposition
• Joint Discriminative-Generative Modeling via Dual Adversarial Training
• Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
• Efficient Edge Test-Time Adaptation via Latent Feature Coordinate Correction
• Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation
• SpurBreast: A Curated Dataset for Investigating Spurious Correlations in Real-world Breast MRI Classification
• Tight Robustness Certificates and Wasserstein Distributional Attacks for Deep Neural Networks
• Probabilistic Variational Contrastive Learning
• Explainable Human-in-the-Loop Segmentation via Critic Feedback Signals
• SynthID-Image: Image watermarking at internet scale
• Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping
• Evaluating the Robustness of a Production Malware Detection System to Transferable Adversarial Attacks
• On the Occurence of Critical Learning Periods in Neural Networks
• Enhancing Visual Prompting through Expanded Transformation Space and Overfitting Mitigation
• Injecting Hallucinations in Autonomous Vehicles: A Component-Agnostic Safety Evaluation Framework
• Revisiting Mixout: An Overlooked Path to Robust Finetuning
• LATTA: Langevin-Anchored Test-Time Adaptation for Enhanced Robustness and Stability
• NEO: No-Optimization Test-Time Adaptation through Latent Re-Centering
• Bridging Reasoning to Learning: Unmasking Illusions using Complexity Out of Distribution Generalization
• Spatially Grounded Concept-Based Image Classification
• Technical note on Sequential Test-Time Adaptation via Martingale-Driven Fisher Prompting
• Technical note on Fisher Information for Robust Federated Cross-Validation
• Neon: Negative Extrapolation From Self-Training Improves Image Generation
• Confidence and Dispersity as Signals: Unsupervised Model Evaluation and Ranking
• Sequence-Preserving Dual-FoV Defense for Traffic Sign and Light Recognition in Autonomous Vehicles
• Quantum Probabilistic Label Refining: Enhancing Label Quality for Robust Image Classification
• VLOD-TTA: Test-Time Adaptation of Vision-Language Object Detectors
• CODED-SMOOTHING: Coding Theory Helps Generalization
• The Impact of Scaling Training Data on Adversarial Robustness
• Annotation-Efficient Active Test-Time Adaptation with Conformal Prediction
• SDPose: Exploiting Diffusion Priors for Out-of-Domain and Robust Pose Estimation
• RDD: Pareto Analysis of the Rate-Distortion-Distinguishability Trade-off
• BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression
• Visual CoT Makes VLMs Smarter but More Fragile
• Evaluating the Impact of Radiographic Noise on Chest X-ray Semantic Segmentation and Disease Classification Using a Scalable Noise Injection Framework
• On the Shelf Life of Fine-Tuned LLM-Judges: Future-Proofing, Backward-Compatibility, and Question Generalization
• Robust Fine-Tuning from Non-Robust Pretrained Models: Mitigating Suboptimal Transfer With Epsilon-Scheduling
• ZeroSiam: An Efficient Siamese for Test-Time Entropy Optimization without Collapse
• TRUST: Test-Time Refinement using Uncertainty-Guided SSM Traverses
• Zubov-Net: Adaptive Stability for Neural ODEs Reconciling Accuracy with Robustness
• POEM: Explore Unexplored Reliable Samples to Enhance Test-Time Adaptation
• TRiCo: Triadic Game-Theoretic Co-Training for Robust Semi-Supervised Learning
• Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracy
• Unlocking Noise-Resistant Vision: Key Architectural Secrets for Robust Models
• Feature Dynamics as Implicit Data Augmentation: A Depth-Decomposed View on Deep Neural Network Generalization
• ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Research
• Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation
• Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
• The Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation From Recognition to Reasoning
• A Validation Strategy for Deep Learning Models: Evaluating and Enhancing Robustness
• MLF-4DRCNet: Multi-Level Fusion with 4D Radar and Camera for 3D Object Detection in Autonomous Driving
• Losing the Plot: How VLM responses degrade on imperfect charts
• Graph Coloring for Multi-Task Learning
• AHA -- Predicting What Matters Next: Online Highlight Detection Without Looking Ahead
• VisMoDAl: Visual Analytics for Evaluating and Improving Corruption Robustness of Vision-Language Models
• FedERL: Federated Efficient and Robust Learning for Common Corruptions
• An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
• Cumulative Consensus Score: Label-Free and Model-Agnostic Evaluation of Object Detectors in Deployment
• The Lifecycle Principle: Stabilizing Dynamic Neural Networks with State Memory
• Post-Hoc Split-Point Self-Consistency Verification for Efficient, Unified Quantification of Aleatoric and Epistemic Uncertainty in Deep Learning
• Promoting Shape Bias in CNNs: Frequency-Based and Contrastive Regularization for Corruption Robustness
• A Modern Look at Simplicity Bias in Image Classification Tasks
• Model-Agnostic Open-Set Air-to-Air Visual Object Detection for Reliable UAV Perception
• ACE and Diverse Generalization via Selective Disagreement
• From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers
• ConstStyle: Robust Domain Generalization with Unified Style Transformation
• Prior Distribution and Model Confidence
• Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation
• Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization
• Differentiable Entropy Regularization: A Complexity-Aware Approach for Neural Optimization
• An Investigation of Visual Foundation Models Robustness
• ADVMEM: Adversarial Memory Initialization for Realistic Test-Time Adaptation via Tracklet-Based Benchmarking
• Domain Adaptation via Feature Refinement
• RT-VLM: Re-Thinking Vision Language Model with 4-Clues for Real-World Object Recognition Robustness
• MorphGen: Morphology-Guided Representation Learning for Robust Single-Domain Generalization in Histopathological Cancer Classification
• Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders
• Entropy-Based Non-Invasive Reliability Monitoring of Convolutional Neural Networks
• Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts
• Active Learning for Neurosymbolic Program Synthesis
• Image Quality Assessment for Machines: Paradigm, Large-scale Database, and Models
• Gradient Rectification for Robust Calibration under Distribution Shift
• ALSA: Anchors in Logit Space for Out-of-Distribution Accuracy Estimation
• Quantization Robustness to Input Degradations for Object Detection
• Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment
• Efficient Multi-Source Knowledge Transfer by Model Merging
• Flatness-aware Curriculum Learning via Adversarial Difficulty
• OASIS: Open-world Adaptive Self-supervised and Imbalanced-aware System
• FOCUS: Frequency-Optimized Conditioning of DiffUSion Models for mitigating catastrophic forgetting during Test-Time Adaptation
• Multi-Plasticity Synergy with Adaptive Mechanism Assignment for Training Spiking Neural Networks
• SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML
• TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyML
• Automated Model Evaluation for Object Detection via Prediction Consistency and Reliability
• Processing and acquisition traces in visual encoders: What does CLIP know about your camera?
• BERTector: An Intrusion Detection Framework Constructed via Joint-dataset Learning Based on Language Model
• On the dynamic evolution of CLIP texture-shape bias and its relationship to human alignment and model robustness
• Harnessing Input-Adaptive Inference for Efficient VLN
• A Guide to Robust Generalization: The Impact of Architecture, Pre-training, and Optimization Strategy
• Never compromise with vulnerabilities: a comprehensive survey on AI governance
• Deep Neural Network Calibration by Reducing Classifier Shift with Stochastic Masking
• Hierarchical Adaptive networks with Task vectors for Test-Time Adaptation
• Adapting Vision-Language Models Without Labels: A Comprehensive Survey
• Keep It Real: Challenges in Attacking Compression-Based Adversarial Purification
• Combined Image Data Augmentations diminish the benefits of Adaptive Label Smoothing
• DocVCE: Diffusion-based Visual Counterfactual Explanations for Document Image Classification
• Slice or the Whole Pie? Utility Control for AI Models
• Selection-Based Vulnerabilities: Clean-Label Backdoor Attacks in Active Learning
• Neutralizing Token Aggregation via Information Augmentation for Efficient Test-Time Adaptation
• The Power of Many: Synergistic Unification of Diverse Augmentations for Efficient Adversarial Robustness
• RegMean++: Enhancing Effectiveness and Generalization of Regression Mean for Model Merging
• Test-Time Model Adaptation for Quantized Neural Networks
• Rein++: Efficient Generalization and Adaptation for Semantic Segmentation with Vision Foundation Models
• Evading Data Provenance in Deep Neural Networks
• Towards Robust Semantic Correspondence: A Benchmark and Insights
• Semantic-Aware Gaussian Process Calibration with Structured Layerwise Kernels for Deep Neural Networks
• Robust 3D Object Detection using Probabilistic Point Clouds from Single-Photon LiDARs
• Beyond topography: Topographic regularization improves robustness and reshapes representations in convolutional neural networks
• From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devices
• COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIP
• Domain Generalization and Adaptation in Intensive Care with Anchor Regression
• Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learning
• Your Attention Matters: to Improve Model Robustness to Noise and Spurious Correlations
• WiSE-OD: Benchmarking Robustness in Infrared Object Detection
• Rainbow Noise: Stress-Testing Multimodal Harmful-Meme Detectors on LGBTQ Content
• CNS-Bench: Benchmarking Image Classifier Robustness Under Continuous Nuisance Shifts
• CLIPTTA: Robust Contrastive Vision-Language Test-Time Adaptation
• Understanding Generalization, Robustness, and Interpretability in Low-Capacity Neural Networks
• Canonical Latent Representations in Conditional Diffusion Models
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks
• Non-exchangeable Conformal Prediction with Optimal Transport: Tackling Distribution Shifts with Unlabeled Data
• Ambient Diffusion Omni: Training Good Models with Bad Data
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection
• Monitoring Risks in Test-Time Adaptation
• SCOOTER: A Human Evaluation Framework for Unrestricted Adversarial Examples
• Where are we with calibration under dataset shift in image classification?
• Divergence-Based Similarity Function for Multi-View Contrastive Learning
• AR2: Attention-Guided Repair for the Robustness of CNNs Against Common Corruptions
• Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge
• Rethinking Detection Calibration: A Coordinate and Direction Perspective
• CTA: Cross-Task Alignment for Better Test Time Training
• Uncertainty-Aware Deepfake Detection via Multi-View Structural Learning
• Addressing The Devastating Effects Of Single-Task Data Poisoning In Exemplar-Free Continual Learning
• FastDINOv2: Frequency Based Curriculum Learning Improves Robustness and Training Speed
• Holistic Continual Learning under Concept Drift with Adaptive Memory Realignment
• How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Tasks
• Out-of-Distribution Detection Methods Answer the Wrong Questions
• Language-Unlocked ViT (LUViT): Empowering Self-Supervised Vision Transformers with LLMs
• Low-Cost Hard-Label Adversarial Attack with Theoretical Foundations
• Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models
• Shared & Domain Self-Adaptive Experts with Frequency-Aware Discrimination for Continual Test-Time Adaptation
• When Small Guides Large: Cross-Model Co-Learning for Test-Time Adaptation
• A New Perspective On AI Safety Through Control Theory Methodologies
• When Test-Time Adaptation Meets Self-Supervised Models
• Time-variant Image Inpainting via Interactive Distribution Transition Estimation
• On the Domain Robustness of Contrastive Vision-Language Models
• Sample Margin-Aware Recalibration of Temperature Scaling
• Smaller = Weaker? Benchmarking Robustness of Quantized LLMs in Code Generation
• Multi-model Online Conformal Prediction with Graph-Structured Feedback
• Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learning
• Quantifying Uncertainty in the Presence of Distribution Shifts
• DRO-Augment Framework: Robustness by Synergizing Wasserstein Distributionally Robust Optimization and Data Augmentation
• Data Curation Matters: Model Collapse and Spurious Shift Performance Prediction from Training on Uncurated Text Embeddings
• LAION-C: An Out-of-Distribution Benchmark for Web-Scale Vision Models
• Understanding the Failure Modes of Out-of-Distribution Generalization
• SyncMapV2: Robust and Adaptive Unsupervised Segmentation
• VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Service
• A Survey on World Models Grounded in Acoustic Physical Information
• GroupNL: Low-Resource and Robust CNN Design over Cloud and Device
• Benchmarking Image Similarity Metrics for Novel View Synthesis Applications
• E-BATS: Efficient Backpropagation-Free Test-Time Adaptation for Speech Foundation Models
• SAFE: Finding Sparse and Flat Minima to Improve Pruning
• Beyond Overconfidence: Foundation Models Redefine Calibration in Deep Neural Networks
• Feature-Based Instance Neighbor Discovery: Advanced Stable Test-Time Adaptation in Dynamic World
• Generalized Incremental Learning under Concept Drift across Evolving Data Streams
• Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks
• Rethinking Semi-supervised Segmentation Beyond Accuracy: Reliability and Robustness
• Enabling Validation for Robust Few-Shot Recognition
• X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP
• Feature-Based Lie Group Transformer for Real-World Applications
• StatsMerging: Statistics-Guided Model Merging via Task-Specific Teacher Distillation
• Test-Time Adaptation with Principal Component Analysis
• Normalize Filters! Classical Wisdom for Deep Vision
• Kernel Embeddings and the Separation of Measure Phenomenon
• Are Synthetic Corruptions A Reliable Proxy For Real-World Corruptions?
• WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales
• SEVA: Leveraging Single-Step Ensemble of Vicinal Augmentations for Test-Time Adaptation
• Pruning Everything, Everywhere, All at Once
• PARC: A Quantitative Framework Uncovering the Symmetries within Vision Language Models
• Test-Time Distillation for Continual Model Adaptation
• PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptation
• Improving Knowledge Distillation Under Unknown Covariate Shift Through Confidence-Guided Data Augmentation
• AVROBUSTBENCH: Benchmarking the Robustness of Audio-Visual Recognition Models at Test-Time
• DiffVQA: Video Quality Assessment Using Diffusion Feature Extractor
• A Flat Minima Perspective on Understanding Augmentations and Model Robustness
• BIRD: Behavior Induction via Representation-structure Distillation
• Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization
• Pseudo Multi-Source Domain Generalization: Bridging the Gap Between Single and Multi-Source Domain Generalization
• Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settings
• Test-time augmentation improves efficiency in conformal prediction
• Test-Time Adaptation of Vision-Language Models for Open-Vocabulary Semantic Segmentation
• From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularization
• CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignment
• Optimizing Data Augmentation through Bayesian Model Selection
• FeatInv: Spatially resolved mapping from feature space to input space using conditional diffusion models
• Frequency Composition for Compressed and Domain-Adaptive Neural Networks
• Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptation
• One-Time Soft Alignment Enables Resilient Learning without Weight Transport
• Variational Deep Learning via Implicit Regularization
• Energy-based Preference Optimization for Test-time Adaptation
• What Is AI Safety? What Do We Want It to Be?
• Curvature Dynamic Black-box Attack: revisiting adversarial robustness via dynamic curvature estimation
• Optimal Conformal Prediction under Epistemic Uncertainty
• Increasing Computation Resolves Conflicts in Vision Language Models
• Distill CLIP (DCLIP): Enhancing Image-Text Retrieval via Cross-Modal Transformer Distillation
• Think Twice before Adaptation: Improving Adaptability of DeepFake Detection via Online Test-Time Adaptation
• S2R-Bench: A Sim-to-Real Evaluation Benchmark for Autonomous Driving
• LORE: Lagrangian-Optimized Robust Embeddings for Visual Encoders
• Test-Time Adaptation with Binary Feedback
• NeuroTrails: Training with Dynamic Sparse Heads as the Key to Effective Ensembling
• SHARDeg: A Benchmark for Skeletal Human Action Recognition in Degraded Scenarios
• CHAOS: Chart Analysis with Outlier Samples
• TULiP: Test-time Uncertainty Estimation via Linearization and Weight Perturbation
• RSC-GestureNet: Reliability-Aware Selective Causal Recognition of Chinese Traffic Police Gestures
• REOBench: Benchmarking Robustness of Earth Observation Foundation Models
• BadDepth: Backdoor Attacks Against Monocular Depth Estimation in the Physical World
• Aggregate-then-Calibrate for Human-centered Assessment with Theoretical Guarantees
• Ranked Entropy Minimization for Continual Test-Time Adaptation
• On the Robustness of Medical Vision-Language Models: Are they Truly Generalizable?
• SNAP: A Benchmark for Testing the Effects of Capture Conditions on Fundamental Vision Tasks
• Fooling the LVLM Judges: Visual Biases in LVLM-Based Evaluation
• Know When to Abstain: Optimal Selective Classification with Likelihood Ratios
• Phonetic Perturbations Reveal Tokenizer-Rooted Safety Gaps in LLMs
• Subtype Robustness Is Not Just Accuracy: Calibration Under Unseen Subtype Shift
• ReservoirTTA: Prolonged Test-time Adaptation for Evolving and Recurring Domains
• Informed Mixing -- Improving Open Set Recognition via Attribution-based Augmentation
• Uniformity First: Uniformity-aware Test-time Adaptation of Vision-language Models against Image Corruption
• Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learning
• Ditch the Denoiser: Emergence of Noise Robustness in Self-Supervised Learning from Data Curriculum
• TDFormer: A Top-Down Attention-Controlled Spiking Transformer
• Are vision language models robust to uncertain inputs?
• MID-L: Matrix-Interpolated Dropout Layer with Layer-wise Neuron Selection
• FRET: Feature Redundancy Elimination for Test Time Adaptation
• Data-Agnostic Augmentations for Unknown Variations: Out-of-Distribution Generalisation in MRI Segmentation
• APCoTTA: Continual Test-Time Adaptation for Semantic Segmentation of Airborne LiDAR Point Clouds
• RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereo
• Efficient Unstructured Pruning of Mamba State-Space Models for Resource-Constrained Environments
• HorusEye: Language as Dynamic Attention for Emergency Visual Analysis
• Fine-Grained Class-Conditional Distribution Balancing for Debiased Learning
• Engineering Risk-Aware, Security-by-Design Frameworks for Assurance of Large-Scale Autonomous AI Models
• DispBench: Benchmarking Disparity Estimation to Synthetic Corruptions
• Multimodal Graph Representation Learning for Robust Surgical Workflow Recognition with Adversarial Feature Disentanglement
• Focal-SAM: Focal Sharpness-Aware Minimization for Long-Tailed Classification
• EmbodiTTA: Resource-Efficient Test-Time Adaptation for Embodied Visual Systems
• Improving Routing in Sparse Mixture of Experts with Graph of Tokens
• Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Test-time Correlation Alignment
• Direct Motion Models for Assessing Generated Videos
• Self-Aware Object Detection via Degradation Manifolds
• A Test Suite for Efficient Robustness Evaluation of Face Recognition Systems
• Supervised Learning Has a Geometric Blind Spot
• Evaluating Test-Time Adaptation For Facial Expression Recognition Under Natural Cross-Dataset Distribution Shifts
• Boosting Single-domain Generalized Object Detection via Vision-Language Knowledge Interaction
• Adversarial Machine Learning-Industry Perspectives
• Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
• Class-Conditional Distribution Balancing for Group Robust Classification
• Evaluating Uncertainty in Deep Gaussian Processes
• Robustness Emerges Early in Training Dynamics, but Is Not Preserved
• Latent Denoising Improves Visual Alignment in Large Multimodal Models
• TestDG: Test-time Domain Generalization for Continual Test-time Adaptation
• Fast Adversarial Training with Weak-to-Strong Spatial-Temporal Consistency in the Frequency Domain on Videos
• Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning
• TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data
• Analysing the Robustness of Vision-Language-Models to Common Corruptions
• Spatial Mixup: Directional Loudness Modification as Data Augmentation for Sound Event Localization and Detection
• Safety Monitoring for Learning-Enabled Cyber-Physical Systems in Out-of-Distribution Scenarios
• Robo-SGG: Exploiting Layout-Oriented Normalization and Restitution Can Improve Robust Scene Graph Generation
• SemDiff: Generating Natural Unrestricted Adversarial Examples via Semantic Attributes Optimization in Diffusion Models
• Iterate or Widen? When Test-Time Refinement Helps LiDAR Scene Completion: A Controlled Study of Evidence Geometry, Training Coverage, and Compute
• Resilience of Vision Transformers for Domain Generalisation in the Presence of Out-of-Distribution Noisy Images
• Invisible Shortcuts: Why Vision Encoders Know Your Camera
• What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective
• NavTrust: Benchmarking Trustworthiness for Embodied Navigation
• Defending Against Frequency-Based Attacks with Diffusion Models
• COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts
• Balancing Two Classifiers via A Simplex ETF Structure for Model Calibration
• BoTTA: Benchmarking on-device Test Time Adaptation
• Self-Bootstrapping for Versatile Test-Time Adaptation
• Evaluating the robustness of explainable AI in medical image recognition under natural and adversarial data corruption
• Foundation model of neural activity predicts response to new stimulus types
• A Meaningful Perturbation Metric for Evaluating Explainability Methods
• Better artificial intelligence does not mean better models of biology
• A Large-Scale Analysis on Contextual Self-Supervised Video Representation Learning
• Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models
• Evaluating and Enhancing Segmentation Model Robustness with Metamorphic Testing
• Deep Positive-Negative Prototypes for Adversarially Robust Discriminative Prototypical Learning
• SPACE: SPike-Aware Consistency Enhancement for Test-Time Adaptation in Spiking Neural Networks
• UAKNN: Label Distribution Learning via Uncertainty-Aware KNN

Related

• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Real World Robustness from Systematic Noise
• A simple way to make neural networks robust against diverse image corruptions
• A Survey of Neural Network Robustness Assessment in Image Recognition
• A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking