SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models

2024/01/16 by Weixiang Zhao, Shilong Wang, Zhao, Weixiang +15 · 6 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2401.08295

Abstract

The continual learning (CL) ability is vital for deploying large language models (LLMs) in the dynamic world. Existing methods devise the learning module to acquire task-specific knowledge with parameter-efficient tuning (PET) block and the selection module to pick out the corresponding one for the testing input, aiming at handling the challenges of catastrophic forgetting and knowledge transfer in CL. However, these methods tend to address only one of the challenges, ignoring the potential of aligning the two modules to effectively address catastrophic forgetting and knowledge transfer simultaneously. To this end, we propose a novel Shared Attention Framework (SAPT), to align the PET learning and selection via the Shared Attentive Learning & Selection module. Extensive Experiments on two CL benchmarks demonstrate the superiority of SAPT. Moreover, SAPT consistently demonstrates its superiority when we scale it to different model sizes (from 770M to 13B), different model architectures (T5 and LLaMA-2) and unseen tasks.

Cited by

• Merge before Forget: A Single LoRA Continual Learning via Continual Merging
• PPSEBM: An Energy-Based Model with Progressive Parameter Selection for Continual Learning
• Parameter Importance-Driven Continual Learning for Foundation Models
• Model Merging Improves Zero-Shot Generalization in Bioacoustic Foundation Models
• SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs
• K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decoupling

Related

• SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models
• Continual Learning of Large Language Models: A Comprehensive Survey
• DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning
• Scalable Language Model with Generalized Continual Learning
• ConPET: Continual Parameter-Efficient Tuning for Large Language Models