Preventing dataset shift from breaking machine-learning biomarkers

2021/07/21 by Dockès, Jéroôme, Varoquaux, Gaël, Poline, Jean-Baptiste · 3 citations
#FOS: Biological sciences #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Quantitative Methods (q-bio.QM) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2107.09947

Abstract

Machine learning brings the hope of finding new biomarkers extracted from cohorts with rich biomedical measurements. A good biomarker is one that gives reliable detection of the corresponding condition. However, biomarkers are often extracted from a cohort that differs from the target population. Such a mismatch, known as a dataset shift, can undermine the application of the biomarker to new individuals. Dataset shifts are frequent in biomedical research, e.g. because of recruitment biases. When a dataset shift occurs, standard machine-learning techniques do not suffice to extract and validate biomarkers. This article provides an overview of when and how dataset shifts breaks machine-learning extracted biomarkers, as well as detection and correction strategies.

Cited by

• Prior-matched evaluation of operational Earth-observation classifiers: a three-number reporting method demonstrated on Sentinel-1 internal-wave detection
• Reliability Assessment Framework Based on Feature Separability for Pathological Cell Image Classification under Prior Bias
• Evaluating Synthetic Data Generation for Domain Generalization in Fetal Brain MRI Segmentation

Related

• Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
• It's easy to fool yourself: Case studies on identifying bias and\n confounding in bio-medical datasets
• How (Not) to Generate a Highly Predictive Biomarker Panel Using Machine Learning
• Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
• Advancements within Modern Machine Learning Methodology: Impacts and Prospects in Biomarker Discovery