The Pessimistic Limits and Possibilities of Margin-based Losses in\n Semi-supervised Learning

2016/12/28 by Jesse H. Krijthe, Krijthe, Jesse H., Marco Loog +1
Computer Science · Mathematics · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.1612.08875

openalex publication_date 2016/12/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Consider a classification problem where we have both labeled and unlabeled\ndata available. We show that for linear classifiers defined by convex\nmargin-based surrogate losses that are decreasing, it is impossible to\nconstruct any semi-supervised approach that is able to guarantee an improvement\nover the supervised classifier measured by this surrogate loss on the labeled\nand unlabeled data. For convex margin-based loss functions that also increase,\nwe demonstrate safe improvements are possible.\n

Related

• Projected Estimators for Robust Semi-supervised Classification
• Unsupervised Supervised Learning II: Training Margin Based Classifiers without Labels
• Semi-supervised learning
• Optimal Binary Classifier Aggregation for General Losses
• Loss factorization, weakly supervised learning and label noise\n robustness