A unifying view on dataset shift in classification

2011/07/23 by Jose G. Moreno-Torres, Troy Raeder, Rocío Alaiz-Rodríguez +3 · 981 citations
Computer Science · Mathematics · #Artificial intelligence #Computer science #Data Stream Mining Techniques #Data science #Epistemology #Field (mathematics) #Imbalanced Data Classification Techniques #Information retrieval #Machine Learning and Data Classification #Mathematics #Natural language processing #Paradigm shift

paper · doi:10.1016/j.patcog.2011.06.019

published in Pattern Recognition 45(1), 521-530 (Elsevier BV)

openalex publication_date 2011/07/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Citations

• Sample Selection Bias as a Specification Error

Cited by

• Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift
• Layer-0 Suppressors Ground Hallucination Inevitability: A Mechanistic Account of How Transformers Trade Factuality for Hedging
• Self-Initiated Open World Learning for Autonomous AI Agents
• Towards Automating the AI Operations Lifecycle
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• Domain Adaptation in Structural Health Monitoring of Civil Infrastructure: A Systematic Review
• Feature-aware Modulation for Learning from Temporal Tabular Data
• Demystifying Feature Engineering in Malware Analysis of API Call Sequences
• An Imbalance-Robust Evaluation Framework for Extreme Risk Forecasts
• Bayesian-based Online Label Shift Estimation with Dynamic Dirichlet Priors
• Ensuring Calibration Robustness in Split Conformal Prediction Under Adversarial Attacks
• TorchXRayVision: A library of chest X-ray datasets and models
• BREEDS: Benchmarks for Subpopulation Shift
• Concept Drift Monitoring and Diagnostics of Supervised Learning Models via Score Vectors
• Brain Graph Super-Resolution Using Adversarial Graph Neural Network with Application to Functional Brain Connectivity
• SurvBench: A Standardised Preprocessing Pipeline for Multi-Modal Electronic Health Record Survival Analysis
• Bot Meets Shortcut: How Can LLMs Aid in Handling Unknown Invariance OOD Scenarios?
• Dataset Safety in Autonomous Driving: Requirements, Risks, and Assurance
• From concept drift to model degradation: An overview on performance-aware drift detectors
• Hidden Incentives for Auto-Induced Distributional Shift
• Cost-sensitive fuzzy rough learning for multi-class imbalanced data classification
• Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods
• ABC-FL: Anomalous and Benign client Classification in Federated Learning
• Unobserved classes and extra variables in high-dimensional discriminant analysis
• Improving Image Classification Robustness through Selective CNN-Filters Fine-Tuning
• Exploring the nature of Galactic unassociated sources detected by the Fermi-LAT
• Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data
• Dos and Don'ts of Machine Learning in Computer Security
• Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
• Enhancing decision support in crop production: Analyzing conformal prediction for uncertainty quantification
• Eigen-Value: Efficient Domain-Robust Data Valuation via Eigenvalue-Based Approach
• Quantifying Feature Importance for Online Content Moderation
• Knowledge Distillation of Uncertainty using Deep Latent Factor Model
• Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition: A Problem-Oriented Perspective
• Geometric Moment Alignment for Domain Adaptation via Siegel Embeddings
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• A Comprehensive Survey of Website Fingerprinting Attacks and Defenses in Tor: Advances and Open Challenges
• Detection of Dataset Shifts in Learning-Enabled Cyber-Physical Systems using Variational Autoencoder for Regression
• Domain-Shift-Aware Conformal Prediction for Large Language Models
• Posterior Re-calibration for Imbalanced Datasets
• Saliency is a Possible Red Herring When Diagnosing Poor Generalization
• Probabilistic Runtime Verification, Evaluation and Risk Assessment of Visual Deep Learning Systems
• An Information-theoretic Approach to Distribution Shifts
• Mutual Information Surprise: Rethinking Unexpectedness in Autonomous Systems
• Discriminative Density-ratio Estimation
• "A 6 or a 9?": Ensemble Learning Through the Multiplicity of Performant Models and Explanations
• Respecting Domain Relations: Hypothesis Invariance for Domain Generalization
• Sequential Drift Detection in Deep Learning Classifiers
• DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift
• Data drift in medical machine learning: implications and potential remedies
• Data Shift of Object Detection in Autonomous Driving
• Domain Agnostic Learning for Unbiased Authentication
• How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy
• Prediction of Survival Outcomes under Clinical Presence Shift: A Joint Neural Network Architecture
• Empirical Evaluation of Concept Drift in ML-Based Android Malware Detection
• Handling Out-of-Distribution Data: A Survey
• Adapting to Fragmented and Evolving Data: A Fisher Information Perspective
• Calibrated Variant Effect Prediction at the Residue Level Using Conditional Score Distributions
• Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection
• Reappraising Domain Generalization in Neural Networks
• Gradient Regularized Contrastive Learning for Continual Domain Adaptation
• Bootstrapped Control Limits for Score-Based Concept Drift Control Charts
• Comparing Behavioural Cloning and Reinforcement Learning for Spacecraft Guidance and Control Networks
• Federated Learning for Commercial Image Sources
• Advances and Open Problems in Federated Learning
• Machine Learning Applications for Therapeutic Tasks with Genomics Data
• Learning to Generate Novel Domains for Domain Generalization
• Mitigating 3D Prostate Biparametric MRI Data Scarcity through Domain Adaptation using Locally-Trained Latent Diffusion Models for Prostate Cancer Detection
• Minimising quantifier variance under prior probability shift
• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• On the limits of cross-domain generalization in automated X-ray prediction
• Concept Drift Detection: Dealing with MissingValues via Fuzzy Distance Estimations
• Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation
• Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs
• Towards a Theoretical Framework of Out-of-Distribution Generalization
• General and Estimable Learning Bound Unifying Covariate and Concept Shifts
• Statistically Valid Post-Deployment Monitoring Should Be Standard for AI-Based Digital Health
• Improving model calibration with accuracy versus uncertainty optimization
• A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability
• Characterizing Fairness Over the Set of Good Models Under Selective Labels
• Bias as a Virtue: Rethinking Generalization under Distribution Shifts
• Targeted Unlearning Using Perturbed Sign Gradient Methods With Applications On Medical Images
• Quantification under prior probability shift: the ratio estimator and its extensions
• MADCAT: Combating Malware Detection Under Concept Drift with Test-Time Adaptation
• Higher-Order Asymptotics of Test-Time Adaptation for Batch Normalization Statistics
• Test for non-negligible adverse shifts
• ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentation
• Mitigating Sampling Bias and Improving Robustness in Active Learning
• Testing for concept shift online
• A Representation Learning Approach to Feature Drift Detection in Wireless Networks
• A taxonomy and review of generalization research in NLP
• Performance Estimation in Binary Classification Using Calibrated Confidence
• Ranking and benchmarking framework for sampling algorithms on synthetic data streams
• Toward Interpretable and Generalizable AI in Regulatory Genomics
• A Deployment Audit of Release-Side Risk in Conformal Triage under Prevalence Shift
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• A survey on concept drift adaptation
• Learning under Concept Drift: A Review
• Challenges in Benchmarking Stream Learning Algorithms with Real-world Data
• Shift Happens: Adjusting Classifiers
• On regularization parameter estimation under covariate shift
• Cross-validation Approaches for Multi-study Predictions
• Characterizing concept drift
• SkillTFM: Gated Skill Evolution for Training-Free Adaptation of Tabular Foundation Models
• Correcting Domain Shifts in Electric Motor Vibration Data for Unseen Operating Conditions
• Task-conditioned Ensemble of Expert Models for Continuous Learning
• LEAP: Linear equations for classifier accuracy prediction under prior probability shift
• Cross-platform normalization of microarray and RNA-seq data for machine learning applications. [europepmc]
• The use of plant models in deep learning: an application to leaf counting in rosette plants. [europepmc]
• Development and Application of a Machine Learning Approach to Assess Short-term Mortality Risk Among Patients With Cancer Starting Chemotherapy. [europepmc]
• Motor Imagery EEG Classification Based on Decision Tree Framework and Riemannian Geometry. [europepmc]
• Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks. [europepmc]
• Causality matters in medical imaging. [europepmc]
• Dynamic memory to alleviate catastrophic forgetting in continual learning with medical imaging. [europepmc]
• A Multi-Level Classification Approach for Sleep Stage Prediction With Processed Data Derived From Consumer Wearable Activity Trackers. [europepmc]
• Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19. [europepmc]
• Deep Learning Classification of Lake Zooplankton. [europepmc]
• Evaluation of domain generalization and adaptation on improving model robustness to temporal dataset shift in clinical medicine. [europepmc]
• Sources of Risk of AI Systems. [europepmc]
• A Unified Framework on Generalizability of Clinical Prediction Models. [europepmc]
• Deploying deep learning models on unseen medical imaging using adversarial domain adaptation. [europepmc]
• DeepAction: a MATLAB toolbox for automated classification of animal behavior in video. [europepmc]
• EHR foundation models improve robustness in the presence of temporal distribution shift. [europepmc]
• Data drift in medical machine learning: implications and potential remedies. [europepmc]
• AI in medical imaging grand challenges: translation from competition to research benefit and patient care. [europepmc]
• Neural network informed photon filtering reduces fluorescence correlation spectroscopy artifacts. [europepmc]
• Why do probabilistic clinical models fail to transport between sites. [europepmc]
• Deep learning to detect left ventricular structural abnormalities in chest X-rays. [europepmc]
• Resilient Artificial Intelligence in Health: Synthesis and Research Agenda Toward Next-Generation Trustworthy Clinical Decision Support. [europepmc]
• Intelligent oncology: The convergence of artificial intelligence and oncology. [europepmc]
• Machine Learning Operations in Health Care: A Scoping Review. [europepmc]
• AI-powered integration of multimodal imaging in precision medicine for neuropsychiatric disorders. [europepmc]