Bi-CoG: Bi-Consistency-Guided Self-Training for Vision-Language Models

2025/10/23 by Zhu, Rui, Lv, Song-Lin, Wang, Zi-Kang +1
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2510.20477

Abstract

Exploiting unlabeled data through semi-supervised learning (SSL) or leveraging pre-trained models via fine-tuning are two prevailing paradigms for addressing label-scarce scenarios. Recently, growing attention has been given to combining fine-tuning of pre-trained vision-language models (VLMs) with SSL, forming the emerging paradigm of semi-supervised fine-tuning. However, existing methods often suffer from model bias and hyperparameter sensitivity, due to reliance on prediction consistency or pre-defined confidence thresholds. To address these limitations, we propose a simple yet effective plug-and-play methodology named \underlineBi-Consistency-\underlineGuided Self-Training (Bi-CoG), which assigns high-quality and low-bias pseudo-labels, by simultaneously exploiting inter-model and intra-model consistency, along with an error-aware dynamic pseudo-label assignment strategy. Both theoretical analysis and extensive experiments over 14 datasets demonstrate the effectiveness of Bi-CoG, which consistently and significantly improves the performance of existing methods.

Citations

• Unlabeled Data vs. Pre-trained Knowledge: Rethinking SSL in the Era of Large Models
• Vision-Language Model Selection and Reuse for Downstream Adaptation
• BMIP: Bi-directional Modality Interaction Prompt Learning for VLM
• ChinaTravel: An Open-Ended Travel Planning Benchmark with Compositional Constraint Validation for Language Agents
• Vision-Language Model Fine-Tuning via Simple Parameter-Efficient Modification
• Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
• Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data
• DeCoOp: Robust Prompt Tuning with Out-of-Distribution Detection
• Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning
• Revisiting Active Learning in the Era of Vision Foundation Models
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting
• Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning
• Overwriting Pretrained Bias with Finetuning Data
• MaPLe: Multi-modal Prompt Learning
• Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
• Unsupervised Prompt Learning for Vision-Language Models
• Conditional Prompt Learning for Vision-Language Models
• Debiased Self-Training for Semi-Supervised Learning
• Debiased Learning from Naturally Imbalanced Pseudo-Labels
• FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling
• The Rich Get Richer: Disparate Impact of Semi-Supervised Learning
• CLIP-Adapter: Better Vision-Language Models with Feature Adapters
• CLIP-Adapter: Better Vision-Language Models with Feature Adapters
• Learning to Prompt for Vision-Language Models
• Learning Transferable Visual Models From Natural Language Supervision
• CoMatch: Semi-supervised Learning with Contrastive Graph Regularization
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification
• Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)
• Describing Textures in the Wild
• Fine-Grained Visual Classification of Aircraft
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

Related

• SelfPrompt: Confidence-Aware Semi-Supervised Tuning for Robust Vision-Language Model Adaptation
• Revisiting semi-supervised learning in the era of foundation models
• Meta Co-Training: Two Views are Better than One
• Simple yet Effective Semi-supervised Knowledge Distillation from Vision-Language Models via Dual-Head Optimization
• Debiased Fine-Tuning for Vision-language Models by Prompt Regularization