Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models

2025/06/09 by Jijie Li, Li, Jijie, Li Du +13 · 17 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.2506.11116

Abstract

Large Language Models (LLMs) demonstrate strong performance in real-world applications, yet existing open-source instruction datasets often concentrate on narrow domains, such as mathematics or coding, limiting generalization and widening the gap with proprietary models. To bridge this gap, we introduce Infinity-Instruct, a high-quality instruction dataset designed to enhance both foundational and chat capabilities of LLMs through a two-phase pipeline. In Phase 1, we curate 7.4M high-quality foundational instructions (InfInstruct-F-7.4M) from over 100M samples using hybrid data selection techniques. In Phase 2, we synthesize 1.5M high-quality chat instructions (InfInstruct-G-1.5M) through a two-stage process involving instruction selection, evolution, and diagnostic filtering. We empirically evaluate Infinity-Instruct by fine-tuning several open-source models, including Mistral, LLaMA, Qwen, and Yi, and observe substantial performance gains across both foundational and instruction following benchmarks, consistently surpassing official instruction-tuned counterparts. Notably, InfInstruct-LLaMA3.1-70B outperforms GPT-4-0314 by 8.6% on instruction following tasks while achieving comparable foundational performance. These results underscore the synergy between foundational and chat training and offer new insights into holistic LLM development. Our dataset\footnotehttps://huggingface.co/datasets/BAAI/Infinity-Instruct and codes\footnotehttps://gitee.com/li-touch/infinity-instruct have been publicly released.

Citations

• Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency
• Aquila2 Technical Report
• Qwen2 Technical Report
• Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
• MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series
• WildChat: 1M ChatGPT Interaction Logs in the Wild
• Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
• LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning
• #InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
• C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models
• Data Selection for Language Models via Importance Resampling
• The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
• Training language models to follow instructions with human feedback
• Training Verifiers to Solve Math Word Problems
• Program Synthesis with Large Language Models
• Evaluating Large Language Models Trained on Code
• Measuring Mathematical Problem Solving With the MATH Dataset
• Measuring Massive Multitask Language Understanding
• Language Models are Few-Shot Learners
• WizardLM: Empowering large pre-trained language models to follow complex instructions

Cited by

• Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data
• ShareChat: A Dataset of Chatbot Conversations in the Wild
• DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI
• LexInstructEval: Lexical Instruction Following Evaluation for Large Language Models
• Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching
• Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models
• When Does Reasoning Matter? A Controlled Study of Reasoning's Contribution to Model Performance
• PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactions
• SpikingBrain: Spiking Brain-inspired Large Models
• FLM-Audio: Natural Monologues Improves Native Full-Duplex Chatbots via Dual Training
• Linear Preference Optimization: Decoupled Gradient Control via Absolute Regularization
• Input-Time Scaling: Adding Noise and Irrelevance into Less-Is-More Drastically Improves Reasoning Performance and Efficiency
• Beyond the Trade-off: Self-Supervised Reinforcement Learning for Reasoning Models' Instruction Following
• RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Services
• Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
• AdaLRS: Loss-Guided Adaptive Learning Rate Search for Efficient Foundation Model Pretraining
• Evaluating Text Creativity across Diverse Domains: A Dataset and Large Language Model Evaluator

Related

• INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
• Instruction Mining: Instruction Data Selection for Tuning Large Language Models
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning
• MAmmoTH2: Scaling Instructions from the Web
• Scaling Instruction-Finetuned Language Models