MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification

2025/06/09 by Sirbu, Iustin, Popovici, Robert-Adrian, Caragea, Cornelia +2 · 1 citation
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #I.2.7 #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2506.07801

Abstract

We introduce MultiMatch, a novel semi-supervised learning (SSL) algorithm combining the paradigms of co-training and consistency regularization with pseudo-labeling. At its core, MultiMatch features a pseudo-label weighting module designed for selecting and filtering pseudo-labels based on head agreement and model confidence, and weighting them according to the perceived classification difficulty. This novel module enhances and unifies three existing techniques -- heads agreement from Multihead Co-training, self-adaptive thresholds from FreeMatch, and Average Pseudo-Margins from MarginMatch -- resulting in a holistic approach that improves robustness and performance in SSL settings. Experimental results on benchmark datasets highlight the superior performance of MultiMatch, i.e., MultiMatch achieves state-of-the-art results on 8 out of 10 setups from 5 natural language processing datasets and ranks first according to the Friedman test among 21 methods. Furthermore, MultiMatch demonstrates exceptional robustness in highly imbalanced settings, outperforming the second-best approach by 3.26%, a critical advantage for real-world text classification tasks. Our code is available on GitHub.

Citations

• CGMatch: A Different Perspective of Semi-supervised Learning
• Co-training for Low Resource Scientific Natural Language Inference
• DeCrisisMB: Debiased Semi-Supervised Learning for Crisis Tweet Classification via Memory Bank
• JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification
• SAT: Improving Semi-Supervised Text Classification with Simple Instance-Adaptive Self-Training
• SimMatch: Semi-supervised Learning with Similarity Matching
• ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised\n Learning
• FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling
• Dash: Semi-Supervised Learning with Dynamic Thresholding
• Semi-Supervised Learning with Multi-Head Co-Training
• AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
• CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
• CoMatch: Semi-supervised Learning with Contrastive Graph Regularization
• Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning
• MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification
• Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response
• Identifying Mislabeled Data using the Area Under the Margin Ranking
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
• Supervised Multimodal Bitransformers for Classifying Images and Text
• Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• Unsupervised Data Augmentation for Consistency Training
• How does Disagreement Help Generalization against Label Corruption?
• Deep Co-Training for Semi-Supervised Image Recognition
• Attention Is All You Need
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
• Yelp Dataset Challenge: Review Rating Prediction
• Character-level Convolutional Networks for Text Classification
• Semi-Supervised Learning with Ladder Networks
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• Semi-Supervised Learning for Large Language Models Safety and Content Moderation

Related

• JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification
• RankingMatch: Delving into Semi-Supervised Learning with Consistency Regularization and Ranking Loss
• RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• DualMatch: Robust Semi-Supervised Learning with Dual-Level Interaction