Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

2022/05/11 by Haokun Liu, Liu, Haokun, Derek Tam +10 · 73 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2205.05638

openalex publication_date 2022/05/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding a small number of training examples as part of the input. ICL incurs substantial computational, memory, and storage costs because it involves processing all of the training examples every time a prediction is made. Parameter-efficient fine-tuning (PEFT) (e.g. adapter modules, prompt tuning, sparse update methods, etc.) offers an alternative paradigm where a small set of parameters are trained to enable a model to perform the new task. In this paper, we rigorously compare few-shot ICL and PEFT and demonstrate that the latter offers better accuracy as well as dramatically lower computational costs. Along the way, we introduce a new PEFT method called (IA)³ that scales activations by learned vectors, attaining stronger performance while only introducing a relatively tiny amount of new parameters. We also propose a simple recipe based on the T0 model called T-Few that can be applied to new tasks without task-specific tuning or modifications. We validate the effectiveness of T-Few on completely unseen tasks by applying it to the RAFT benchmark, attaining super-human performance for the first time and outperforming the state-of-the-art by 6% absolute. All of the code used in our experiments is publicly available.

Cited by

• The Appeal and Reality of Recycling LoRAs with Adaptive Merging
• Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias
• RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks
• Fine-Tuned In-Context Learners for Efficient Adaptation
• Ladder Up, Memory Down: Low-Cost Fine-Tuning With Side Nets
• PVeRA: Probabilistic Vector-Based Random Matrix Adaptation
• PlantBiMoE: A Bidirectional Foundation Model with SparseMoE for Plant Genomes
• Group Orthogonal Low-Rank Adaptation for RGB-T Tracking
• RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning
• PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models
• Serving Heterogeneous LoRA Adapters in Distributed LLM Inference Systems
• FedSGT: Exact Federated Unlearning via Sequential Group-based Training
• PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark
• Delta Sampling: Data-Free Knowledge Transfer Across Diffusion Models
• Advanced Black-Box Tuning of Large Language Models with Limited API Calls
• Selecting Auxiliary Data via Neural Tangent Kernels for Low-Resource Domains
• LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
• Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation
• QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Code
• On The Dangers of Poisoned LLMs In Security Automation
• Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs
• Languages are Modalities: Cross-Lingual Alignment via Encoder Injection
• Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
• Will Scaling Improve Social Simulation with LLMs?
• MIN-Merging: Merge the Important Neurons for Model Merging
• Instant Personalized Large Language Model Adaptation via Hypernetwork
• From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning
• Rethinking Visual Intelligence: Insights from Video Pretraining
• Large language model-based task planning for service robots: A review
• Low-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-Study
• Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
• Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples
• Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models
• QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs
• Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models
• FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
• SliceFine: The Universal Winning-Slice Hypothesis for Pretrained Networks
• AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent System
• BOTANIC-0: a series of foundation models for plant genomic data
• FT-MDT: Extracting Decision Trees from Medical Texts via a Novel Low-rank Adaptation Method
• Learning from Convenience Samples: A Case Study on Fine-Tuning LLMs for Survey Non-response in the German Longitudinal Election Study
• Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey
• F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning
• IA2: Alignment with ICL Activations Improves Supervised Fine-Tuning
• Stability of In-Context Learning: A Spectral Coverage Perspective
• Tight Sample Complexity for Low-Rank Adaptation: Matching Bounds and Rank Selection
• Transporting Task Vectors across Different Architectures without Training
• SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
• HyperAdapt: Simple High-Rank Adaptation
• Weights-Rotated Preference Optimization for Large Language Models
• TASO: Task-Aligned Sparse Optimization for Parameter-Efficient Model Adaptation
• QWHA: Quantization-Aware Walsh-Hadamard Adaptation for Parameter-Efficient Fine-Tuning on Large Language Models
• When Big Models Train Small Ones: Label-Free Model Parity Alignment for Efficient Visual Question Answering using Small VLMs
• BEFT: Bias-Efficient Fine-Tuning of Language Models
• Distribution-Aligned Decoding for Efficient LLM Task Adaptation
• How Can Quantum Deep Learning Improve Large Language Models?
• Black-box Model Merging for Language-Model-as-a-Service with Massive Model Repositories
• The Anatomy of Alignment: Decomposing Preference Optimization by Steering Sparse Features
• A Systematic Evaluation of Parameter-Efficient Fine-Tuning Methods for the Security of Code LLMs
• Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
• HEFT: A Coarse-to-Fine Hierarchy for Enhancing the Efficiency and Accuracy of Language Model Reasoning
• Language-Guided Long Horizon Manipulation with LLM-based Planning and Visual Perception
• MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper
• Can Smaller LLMs do better? Unlocking Cross-Domain Potential through Parameter-Efficient Fine-Tuning for Text Summarization
• EZ-Sort: Efficient Pairwise Comparison via Zero-Shot CLIP-Based Pre-Ordering and Human-in-the-Loop Sorting
• It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs
• ChronoLLM: Customizing Language Models for Physics-Based Simulation Code Generation
• Cross-Prompt Encoder for Low-Performing Languages
• Improving Learning of New Diseases through Knowledge-Enhanced Initialization for Federated Adapter Tuning
• A Data-driven ML Approach for Maximizing Performance in LLM-Adapter Serving
• Align, Don't Divide: Revisiting the LoRA Architecture in Multi-Task Learning
• Leveraging large language models for SQL behavior-based database intrusion detection
• Role-Aware Language Models for Secure and Contextualized Access Control in Organizations

Related

• Making Pre-trained Language Models Better Few-shot Learners
• Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation
• Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
• Optimization-Inspired Few-Shot Adaptation for Large Language Models
• Scaled Prompt-Tuning for Few-Shot Natural Language Generation