2021/11/10 by Benoît Dufumier, Dufumier, Benoit, Pietro Gori +7
Computer Science · Engineering · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Geophysical Methods and Applications #Machine Learning (cs.LG) #Sparse and Compressive Sensing Techniques
paper · pdf · doi:10.48550/arxiv.2111.05643
openalex publication_date 2021/11/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Contrastive Learning has shown impressive results on natural and medical\nimages, without requiring annotated data. However, a particularity of medical\nimages is the availability of meta-data (such as age or sex) that can be\nexploited for learning representations. Here, we show that the recently\nproposed contrastive y-Aware InfoNCE loss, that integrates multi-dimensional\nmeta-data, asymptotically optimizes two properties: conditional alignment and\nglobal uniformity. Similarly to [Wang, 2020], conditional alignment means that\nsimilar samples should have similar features, but conditionally on the\nmeta-data. Instead, global uniformity means that the (normalized) features\nshould be uniformly distributed on the unit hyper-sphere, independently of the\nmeta-data. Here, we propose to define conditional uniformity, relying on the\nmeta-data, that repel only samples with dissimilar meta-data. We show that\ndirect optimization of both conditional alignment and uniformity improves the\nrepresentations, in terms of linear evaluation, on both CIFAR-100 and a brain\nMRI dataset.\n