Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency

2024/09/11 by H. Zhao, Zhao, Hanyu, Lidong Du +7 · 6 citations
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Education and Learning Interventions #FOS: Computer and information sciences #Online Learning and Analytics #Online and Blended Learning

paper · pdf · doi:10.48550/arxiv.2409.07045

openalex publication_date 2024/09/11 · openalex created_date 2024/10/22 · openalex updated_date 2026/07/28

Abstract

With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs). Previous research mainly focuses on selecting individual high-quality instructions. However, these works overlooked the joint interactions and dependencies between different categories of instructions, leading to suboptimal selection strategies. Moreover, the nature of these interaction patterns remains largely unexplored, let alone optimize the instruction set with regard to them. To fill these gaps, in this paper, we: (1) systemically investigate interaction and dependency patterns between different categories of instructions, (2) manage to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning. Experimental results across different LLMs demonstrate improved performance over strong baselines on widely adopted benchmarks.

Cited by

• The PLLuM Instruction Corpus
• EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models
• PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactions
• Accelerate Scaling of LLM Finetuning via Quantifying the Coverage and Depth of Instruction Set
• FLM-Audio: Natural Monologues Improves Native Full-Duplex Chatbots via Dual Training
• Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models

Related

• RAISE: Reinforced Adaptive Instruction Selection For Large Language Models
• InstructEval: Systematic Evaluation of Instruction Selection Methods
• Smaller Language Models Are Better Instruction Evolvers
• Instruction Learning Paradigms: A Dual Perspective on White-box and Black-box LLMs
• A Survey on Data Selection for LLM Instruction Tuning