MoTiC: Momentum Tightness and Contrast for Few-Shot Class-Incremental Learning

2025/09/24 by He, Zeyu, Huang, Shuai, Lu, Yuwu +1
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2509.19664

Abstract

Few-Shot Class-Incremental Learning (FSCIL) must contend with the dual challenge of learning new classes from scarce samples while preserving old class knowledge. Existing methods use the frozen feature extractor and class-averaged prototypes to mitigate against catastrophic forgetting and overfitting. However, new-class prototypes suffer significant estimation bias due to extreme data scarcity, whereas base-class prototypes benefit from sufficient data. In this work, we theoretically demonstrate that aligning the new-class priors with old-class statistics via Bayesian analysis reduces variance and improves prototype accuracy. Furthermore, we propose large-scale contrastive learning to enforce cross-category feature tightness. To further enrich feature diversity and inject prior information for new-class prototypes, we integrate momentum self-supervision and virtual categories into the Momentum Tightness and Contrast framework (MoTiC), constructing a feature space with rich representations and enhanced interclass cohesion. Experiments on three FSCIL benchmarks produce state-of-the-art performances, particularly on the fine-grained task CUB-200, validating our method's ability to reduce estimation bias and improve incremental learning robustness.

Citations

• CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning
• Compositional Few-Shot Class-Incremental Learning
• OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning
• Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning
• A Survey on Few-Shot Class-Incremental Learning
• Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning
• Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning
• Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation
• Few-Shot Class-Incremental Learning from an Open-Set Perspective
• Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks
• Forward Compatible Few-Shot Class-Incremental Learning
• Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima
• Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting
• Few-Shot Incremental Learning with Continually Evolved Classifiers
• A Broad Study on the Transferability of Visual Representations with Contrastive Learning
• Learning Transferable Visual Models From Natural Language Supervision
• How Well Do Self-Supervised Models Transfer?
• Exploring Simple Siamese Representation Learning
• Exploring Simple Siamese Representation Learning
• Why Do Better Loss Functions Lead to Less Transferable Features?
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Bootstrap your own latent: A new approach to self-supervised Learning
• Few-Shot Class-Incremental Learning
• Supervised Contrastive Learning
• Negative Margin Matters: Understanding Margin in Few-shot Classification
• A Simple Framework for Contrastive Learning of Visual Representations
• Momentum Contrast for Unsupervised Visual Representation Learning
• Momentum Contrast for Unsupervised Visual Representation Learning
• Similarity of Neural Network Representations Revisited
• Deep Clustering for Unsupervised Learning of Visual Features
• Do Better ImageNet Models Transfer Better?
• Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
• Matching Networks for One Shot Learning
• Learning Deep Features for Discriminative Localization
• Deep Residual Learning for Image Recognition

Related

• CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning
• Bias Mitigating Few-Shot Class-Incremental Learning
• Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning
• Incremental Few-Shot Learning via Implanting and Compressing
• Rethinking Few-shot Class-incremental Learning: Learning from Yourself