On Weak-to-Strong Generalization and f-Divergence

2025/06/03 by Wei Yao, Gengze Xu, Yao, Wei +11 · 1 citation
Computer Science · Decision Sciences · Mathematics · #Approximation Theory and Sequence Spaces #FOS: Computer and information sciences #Fuzzy and Soft Set Theory #Machine Learning (cs.LG) #Optimization and Variational Analysis

paper · pdf · doi:10.48550/arxiv.2506.03109

openalex publication_date 2025/06/03 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

Weak-to-strong generalization (W2SG) has emerged as a promising paradigm for stimulating the capabilities of strong pre-trained models by leveraging supervision from weaker supervisors. To improve the performance of the strong model, existing methods often require additional weak models or complex procedures, leading to substantial computational and memory overhead. Motivated by the effectiveness of f-divergence loss in various machine learning domains, we introduce f-divergence as an information-theoretic loss function framework in W2SG. Our theoretical analysis reveals fundamental limitations and equivalence of different f-divergence losses in W2SG, supported by sample complexity bounds and information-theoretic insights. We empirically demonstrate that f-divergence loss, which generalizes widely-used metrics like KL divergence, effectively improves generalization and noise tolerance of the strong model in practice.

Cited by

• Weak-to-Strong On-Policy Distillation

Related

• The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibration
• On the Emergence of Weak-to-Strong Generalization: A Bias-Variance Perspective
• Revisiting Weak-to-Strong Generalization in Theory and Practice: Reverse KL vs. Forward KL
• Relating Misfit to Gain in Weak-to-Strong Generalization Beyond the Squared Loss
• Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions