2021/04/19 by Mandy Lu, Qingyu Zhao, Lu, Mandy +11 · 1 citation
Computer Science · #AI in cancer detection #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Face recognition and analysis #Machine Learning (cs.LG) #cs.LG
paper · pdf · doi:10.48550/arxiv.2104.09052
Accepted to CVPR 2021. Project page: https://mml.stanford.edu/MDN/
openalex publication_date 2021/04/19 · arxiv created 2021/05/05 · arxiv updated 2021/05/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Batch Normalization (BN) and its variants have delivered tremendous success in combating the covariate shift induced by the training step of deep learning methods. While these techniques normalize feature distributions by standardizing with batch statistics, they do not correct the influence on features from extraneous variables or multiple distributions. Such extra variables, referred to as metadata here, may create bias or confounding effects (e.g., race when classifying gender from face images). We introduce the Metadata Normalization (MDN) layer, a new batch-level operation which can be used end-to-end within the training framework, to correct the influence of metadata on feature distributions. MDN adopts a regression analysis technique traditionally used for preprocessing to remove (regress out) the metadata effects on model features during training. We utilize a metric based on distance correlation to quantify the distribution bias from the metadata and demonstrate that our method successfully removes metadata effects on four diverse settings: one synthetic, one 2D image, one video, and one 3D medical image dataset.