LST: Lexicon-Guided Self-Training for Few-Shot Text Classification

2022/02/05 by Kim, Hazel, Son, Jaeman, Han, Yo-Sub
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2202.02566

Abstract

Self-training provides an effective means of using an extremely small amount of labeled data to create pseudo-labels for unlabeled data. Many state-of-the-art self-training approaches hinge on different regularization methods to prevent overfitting and improve generalization. Yet they still rely heavily on predictions initially trained with the limited labeled data as pseudo-labels and are likely to put overconfident label belief on erroneous classes depending on the first prediction. To tackle this issue in text classification, we introduce LST, a simple self-training method that uses a lexicon to guide the pseudo-labeling mechanism in a linguistically-enriched manner. We consistently refine the lexicon by predicting confidence of the unseen data to teach pseudo-labels better in the training iterations. We demonstrate that this simple yet well-crafted lexical knowledge achieves 1.0-2.0% better performance on 30 labeled samples per class for five benchmark datasets than the current state-of-the-art approaches.

Related

• Uncertainty-aware Self-training for Text Classification with Few Labels
• Zero-Shot Text Classification with Self-Training
• Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets
• SAT: Improving Semi-Supervised Text Classification with Simple Instance-Adaptive Self-Training
• Boosting Prompt-Based Self-Training With Mapping-Free Automatic Verbalizer for Multi-Class Classification