From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning

2023/08/23 by Ming Li, Li, Ming, Yong Zhang +15 · 54 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2308.12032

openalex publication_date 2023/08/23 · openalex created_date 2023/08/25 · openalex updated_date 2026/07/28

Abstract

In the realm of Large Language Models (LLMs), the balance between instruction data quality and quantity is a focal point. Recognizing this, we introduce a self-guided methodology for LLMs to autonomously discern and select cherry samples from open-source datasets, effectively minimizing manual curation and potential cost for instruction tuning an LLM. Our key innovation, the Instruction-Following Difficulty (IFD) metric, emerges as a pivotal metric to identify discrepancies between a model's expected responses and its intrinsic generation capability. Through the application of IFD, cherry samples can be pinpointed, leading to a marked uptick in model training efficiency. Empirical validations on datasets like Alpaca and WizardLM underpin our findings; with a mere 10% of original data input, our strategy showcases improved results. This synthesis of self-guided cherry-picking and the IFD metric signifies a transformative leap in the instruction tuning of LLMs, promising both efficiency and resource-conscious advancements. Codes, data, and models are available: https://github.com/tianyi-lab/Cherry_LLM

Cited by

• Accounting Reasoning in Large Language Models: Concepts, Evaluation, and Empirical Analysis
• Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy
• Schoenfeld's Anatomy of Mathematical Reasoning by Language Models
• SiamGPT: Quality-First Fine-Tuning for Stable Thai Text Generation
• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value
• AIR: Post-training Data Selection for Reasoning via Attention Head Influence
• T-pro 2.0: An Efficient Russian Hybrid-Reasoning Model and Playground
• Towards Active Synthetic Data Generation for Finetuning Language Models
• Importance-Aware Data Selection for Efficient LLM Instruction Tuning
• Selecting Auxiliary Data via Neural Tangent Kernels for Low-Resource Domains
• Synergy over Discrepancy: A Partition-Based Approach to Multi-Domain LLM Fine-Tuning
• Evontree: Ontology Rule-Guided Self-Evolution of Large Language Models
• A Survey on Efficient Large Language Model Training: From Data-centric Perspectives
• DCMM-SQL: Automated Data-Centric Pipeline and Multi-Model Collaboration Training for Text-to-SQL Model
• AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generation
• CircuitSeer: Mining High-Quality Data by Probing Mathematical Reasoning Circuits in LLMs
• Computational Budget Should Be Considered in Data Selection
• Holdout-Loss-Based Data Selection for LLM Finetuning via In-Context Learning
• Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
• CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization
• TRIM: Token-wise Attention-Derived Saliency for Data-Efficient Instruction Tuning
• Increasing LLM response trustworthiness using voting ensembles
• Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
• Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment
• Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration
• On Code-Induced Reasoning in LLMs
• A method for improving multilingual quality and diversity of instruction fine-tuning datasets
• Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory
• Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning
• Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
• Active Domain Knowledge Acquisition with 100-Dollar Budget: Enhancing LLMs via Cost-Efficient, Expert-Involved Interaction in Sensitive Domains
• Accelerate Scaling of LLM Finetuning via Quantifying the Coverage and Depth of Instruction Set
• GRAM-R²: Self-Training Generative Foundation Reward Models for Reward Reasoning
• DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
• Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning
• Intern-S1: A Scientific Multimodal Foundation Model
• ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning
• Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation
• Data Selection for LLM Alignment Using Fine-Grained Preferences
• Δ-AttnMask: Attention-Guided Masked Hidden States for Efficient Data Selection and Augmentation
• AGORA: Incentivizing Group Emergence Capability in LLMs via Group Distillation
• Distributional Machine Unlearning via Selective Data Removal
• Disentangling the Roles of Representation and Selection in Data Pruning
• TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection
• DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
• Efficient Code LLM Training via Distribution-Consistent and Diversity-Aware Data Selection
• MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
• CLUES: Collaborative High-Quality Data Selection for LLMs via Training Dynamics
• Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models
• HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models
• SFT-GO: Supervised Fine-Tuning with Group Optimization for Large Language Models
• LearnAlign: Reasoning Data Selection for Reinforcement Learning in Large Language Models Based on Improved Gradient Alignment
• What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding
• Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models

Related

• Instruction Mining: Instruction Data Selection for Tuning Large Language Models
• Maybe Only 0.5% Data is Needed: A Preliminary Exploration of Low Training Data Instruction Tuning
• A Survey on Data Selection for LLM Instruction Tuning
• LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning
• Importance-Aware Data Selection for Efficient LLM Instruction Tuning