Handling Out-of-Distribution Data: A Survey
2025/07/25 by Tamang, Lakpa, Bouadjenek, Mohamed Reda, Dazeley, Richard +1 · 3 citations
#68T07 (Primary) #68T10 (Secondary) #68T45 #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #I.5.1 #Machine Learning (cs.LG)
paper · doi:10.48550/arxiv.2507.21160
Abstract
In the field of Machine Learning (ML) and data-driven applications, one of the significant challenge is the change in data distribution between the training and deployment stages, commonly known as distribution shift. This paper outlines different mechanisms for handling two main types of distribution shifts: (i) Covariate shift: where the value of features or covariates change between train and test data, and (ii) Concept/Semantic-shift: where model experiences shift in the concept learned during training due to emergence of novel classes in the test phase. We sum up our contributions in three folds. First, we formalize distribution shifts, recite on how the conventional method fails to handle them adequately and urge for a model that can simultaneously perform better in all types of distribution shifts. Second, we discuss why handling distribution shifts is important and provide an extensive review of the methods and techniques that have been developed to detect, measure, and mitigate the effects of these shifts. Third, we discuss the current state of distribution shift handling mechanisms and propose future research directions in this area. Overall, we provide a retrospective synopsis of the literature in the distribution shift, focusing on OOD data that had been overlooked in the existing surveys.
Citations
- Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts
- Margin-bounded Confidence Scores for Out-of-Distribution Detection
- Enhancing Domain Adaptation through Prompt Gradient Alignment
- Soft Prompt Generation for Domain Generalization
- Rethinking Multi-domain Generalization with A General Learning Objective
- Subspace Identification for Multi-Source Domain Adaptation
- Meta OOD Learning for Continuously Adaptive OOD Detection
- PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization
- Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection
- Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations
- Peer-to-Peer Federated Continual Learning for Naturalistic Driving Action Recognition
- Continual Semantic Segmentation with Automatic Memory Sample Selection
- FREDOM: Fairness Domain Adaptation Approach to Semantic Scene Understanding
- Astroformer: More Data Might not be all you need for Classification
- OpenMix: Exploring Outlier Samples for Misclassification Detection
- Mind the Label Shift of Augmentation-based Graph OOD Generalization
- Sharpness-Aware Gradient Matching for Domain Generalization
- Progressive Open Space Expansion for Open-Set Model Attribution
- Annotated Point Clouds and Images from a Spatial-Semantic Perception Pipeline: Robotized Deconstruction at the Reference Construction Site, Aachen
- Diversity-Measurable Anomaly Detection
- On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective
- Derandomized Novelty Detection with FDR Control via Conformal E-values
- RLSbench: Domain Adaptation Under Relaxed Label Shift
- Energy-based Out-of-Distribution Detection for Graph Neural Networks
- Rethinking Out-of-distribution (OOD) Detection: Masked Image Modeling is All You Need
- Progressive Prompts: Continual Learning for Language Models
- Pushing the Limits of Fewshot Anomaly Detection in Industry Vision: Graphcore
- GLIGEN: Open-Set Grounded Text-to-Image Generation
- Sparse Coding in a Dual Memory System for Lifelong Learning
- Runtime Monitoring for Out-of-Distribution Detection in Object Detection Neural Networks
- PIVOT: Prompting for Video Continual Learning
- MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation
- DATID-3D: Diversity-Preserved Domain Adaptation Using Text-to-Image Diffusion for 3D Generative Model
- MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly Detection
- Denoising diffusion models for out-of-distribution detection
- Is Out-of-Distribution Detection Learnable?
- Federated Domain Generalization for Image Recognition via Cross-Client Style Transfer
- Visual Prompt Tuning for Generative Transfer Learning
- Domain Adaptation under Open Set Label Shift
- Online Continual Learning with Contrastive Vision Transformer
- Class-Specific Semantic Reconstruction for Open Set Recognition
- An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems
- Mitigating Neural Network Overconfidence with Logit Normalization
- Continual Learning with Bayesian Model based on a Fixed Pre-trained Feature Extractor
- NICO++: Towards Better Benchmarking for Domain Generalization
- What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?
- Full-Spectrum Out-of-Distribution Detection
- A Closer Look at Rehearsal-Free Continual Learning
- Causality Inspired Representation Learning for Domain Generalization
- Bamboo: Building Mega-Scale Vision Dataset Continually with Human-Machine Synergy
- Domain Adaptation via Prompt Learning
- Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs
- Training OOD Detectors in their Natural Habitats
- PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
- Provable Guarantees for Understanding Out-of-distribution Detection
- ReAct: Out-of-distribution Detection With Rectified Activations
- Towards Open-Set Text Recognition via Label-to-Prototype Learning
- Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data
- Generalized Out-of-Distribution Detection: A Survey
- Generalized Out-of-Distribution Detection: A Survey
- Spatial Location Constraint Prototype Loss for Open Set Recognition
- On the Importance of Gradients for Detecting Distributional Shifts in the Wild
- Learning to Prompt for Vision-Language Models
- Towards Out-Of-Distribution Generalization: A Survey
- An Adaptable Deep Learning-Based Intrusion Detection System to Zero-Day Attacks
- Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions
- Co2L: Contrastive Continual Learning
- Examining and Combating Spurious Features under Distribution Shift
- Towards a Theoretical Framework of Out-of-Distribution Generalization
- Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environments
- Open-world Machine Learning: Applications, Challenges, and Opportunities
- Masked Contrastive Learning for Anomaly Detection
- New Insights on Reducing Abrupt Representation Change in Online Continual Learning
- New Insights on Reducing Abrupt Representation Change in Online Continual Learning
- OpenGAN: Open-Set Recognition via Open Data Generation
- Understanding Continual Learning Settings with Data Distribution Drift Analysis
- Learning Placeholders for Open-Set Recognition
- Incorporating Convolution Designs into Visual Transformers
- SWAD: Domain Generalization by Seeking Flat Minima
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation
- Assured Autonomy: Path Toward Living With Autonomous Systems We Can Trust
- Energy-based Out-of-distribution Detection
- A Brief Review of Domain Adaptation
- Sharpness-Aware Minimization for Efficiently Improving Generalization
- Driver Anomaly Detection: A Dataset and Contrastive Learning Approach
- Batch Normalization Embeddings for Deep Domain Generalization
- Transfer Learning in Deep Reinforcement Learning: A Survey
- Zero Shot Domain Generalization
- CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
- In Search of Lost Domain Generalization
- ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining
- Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?
- ESL: Entropy-guided Self-supervised Learning for Domain Adaptation in Semantic Segmentation
- Dark Experience for General Continual Learning: a Strong, Simple Baseline
- FDA: Fourier Domain Adaptation for Semantic Segmentation
- Gradually Vanishing Bridge for Adversarial Domain Adaptation
- Domain-aware Visual Bias Eliminating for Generalized Zero-Shot Learning
- Learning to Learn Single Domain Generalization
- Out-of-Distribution Generalization via Risk Extrapolation (REx)
- Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data
- Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation
- A Simple Framework for Contrastive Learning of Visual Representations
- Big Transfer (BiT): General Visual Representation Learning
- AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
- A Comprehensive Survey on Transfer Learning
- A Comprehensive Survey on Transfer Learning
- Deep Weakly-supervised Anomaly Detection
- Deep Anomaly Detection with Deviation Networks
- Invariant Risk Minimization
- Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
- Deep Semi-Supervised Anomaly Detection
- Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
- Semi-supervised Domain Adaptation via Minimax Entropy
- C2AE: Class Conditioned Auto-Encoder for Open-set Recognition
- Improving Neural Architecture Search Image Classifiers via Ensemble Learning
- Contrastive Adaptation Network for Unsupervised Domain Adaptation
- Task-Free Continual Learning
- Experience Replay for Continual Learning
- Deep Anomaly Detection with Outlier Exposure
- Metric Learning for Novelty and Anomaly Detection
- End-to-End Incremental Learning
- Latent Space Autoregression for Novelty Detection
- Uncertainty in the Variational Information Bottleneck
- Progress & Compress: A scalable framework for continual learning
- Taskonomy: Disentangling Task Transfer Learning
- Adversarially Learned One-Class Classifier for Novelty Detection
- Incremental Adversarial Domain Adaptation for Continually Changing\n Environments
- mixup: Beyond Empirical Risk Minimization
- Unified Deep Supervised Domain Adaptation and Generalization
- Partial Transfer Learning with Selective Adversarial Networks
- Generative OpenMax for Multi-Class Open Set Classification
- Sparse Representation-based Open Set Recognition
- iCaRL: Incremental Classifier and Representation Learning
- Learning the Roots of Visual Domain Shift
- Deep Transfer Learning with Joint Adaptation Networks
- Towards Open Set Deep Networks
- Return of Frustratingly Easy Domain Adaptation
- Unsupervised Domain Adaptation by Backpropagation
- Return of the Devil in the Details: Delving Deep into Convolutional Nets
- Intriguing properties of neural networks
- ImageNet classification with deep convolutional neural networks
- A Survey on Transfer Learning
- An overview of statistical learning theory
- The perceptron: A probabilistic model for information storage and organization in the brain.
- Combining inherent knowledge of vision-language models with unsupervised domain adaptation through strong-weak guidance
- A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifiers
- A unifying view on dataset shift in classification
Cited by
Related