2021/02/17 by Kai Sheng Tai, Tai, Kai Sheng, Peter Bailis +3 · 7 citations
Computer Science · Mathematics · #Advanced Neural Network Applications #Algorithm #Artificial intelligence #Class (philosophy) #Classifier (UML) #Code (set theory) #Computer science #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Machine learning #Mathematical optimization #Mathematics #Natural Language Processing Techniques #Neural Networks and Applications #Simulated annealing #Training set #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2102.08622
published in arXiv (Cornell University), 10065-10075 (Cornell University) · ICML 2021 camera ready version
openalex publication_date 2021/02/17 · arxiv created 2021/06/12 · arxiv updated 2021/06/15 · openalex created_date 2022/07/25 · openalex updated_date 2026/08/05
Self-training is a standard approach to semi-supervised learning where the learner's own predictions on unlabeled data are used as supervision during training. In this paper, we reinterpret this label assignment process as an optimal transportation problem between examples and classes, wherein the cost of assigning an example to a class is mediated by the current predictions of the classifier. This formulation facilitates a practical annealing strategy for label assignment and allows for the inclusion of prior knowledge on class proportions via flexible upper bound constraints. The solutions to these assignment problems can be efficiently approximated using Sinkhorn iteration, thus enabling their use in the inner loop of standard stochastic optimization algorithms. We demonstrate the effectiveness of our algorithm on the CIFAR-10, CIFAR-100, and SVHN datasets in comparison with FixMatch, a state-of-the-art self-training algorithm. Our code is available at https://github.com/stanford-futuredata/sinkhorn-label-allocation.