Self-training Large Language Models through Knowledge Detection

2024/06/17 by Wei Jie Yeo, Yeo, Wei Jie, Teddy Ferdinan +7 · 3 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2406.11275

openalex publication_date 2024/06/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) often necessitate extensive labeled datasets and training compute to achieve impressive performance across downstream tasks. This paper explores a self-training paradigm, where the LLM autonomously curates its own labels and selectively trains on unknown data samples identified through a reference-free consistency method. Empirical evaluations demonstrate significant improvements in reducing hallucination in generation across multiple subjects. Furthermore, the selective training framework mitigates catastrophic forgetting in out-of-distribution benchmarks, addressing a critical limitation in training LLMs. Our findings suggest that such an approach can substantially reduce the dependency on large labeled datasets, paving the way for more scalable and cost-effective language model training.

Cited by

• GRAM-R²: Self-Training Generative Foundation Reward Models for Reward Reasoning
• MeMo: Memory as a Model
• LLM-Evaluation Tropes: Perspectives on the Validity of LLM-Evaluations
• SEA-LION: Southeast Asian Languages in One Network

Related

• Into the Unknown: Self-Learning Large Language Models
• Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression
• Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data
• Augmenting LLMs with Knowledge: A survey on hallucination prevention
• SPaRFT: Self-Paced Reinforcement Fine-Tuning for Large Language Models