Contrastive Learning with Nasty Noise

2025/02/25 by Zhao, Ziruo
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2502.17872

Abstract

Contrastive learning has emerged as a powerful paradigm for self-supervised representation learning. This work analyzes the theoretical limits of contrastive learning under nasty noise, where an adversary modifies or replaces training samples. Using PAC learning and VC-dimension analysis, lower and upper bounds on sample complexity in adversarial settings are established. Additionally, data-dependent sample complexity bounds based on the l2-distance function are derived.

Related

• Generalization Bounds for Robust Contrastive Learning: From Theory to Practice
• Generalization Bounds for Adversarial Contrastive Learning
• Adversarial Self-Supervised Contrastive Learning
• Towards Efficient Contrastive PAC Learning
• Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness