Instruction Tuning with GPT-4

2023/04/06 by Baolin Peng, Peng, Baolin, Chunyuan Li +7 · 3 voices · 40 citations
Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2304.03277

openalex publication_date 2023/04/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed. In this paper, we present the first attempt to use GPT-4 to generate instruction-following data for LLM finetuning. Our early experiments on instruction-tuned LLaMA models show that the 52K English and Chinese instruction-following data generated by GPT-4 leads to superior zero-shot performance on new tasks to the instruction-following data generated by previous state-of-the-art models. We also collect feedback and comparison data from GPT-4 to enable a comprehensive evaluation and reward model training. We make our data generated using GPT-4 as well as our codebase publicly available.

Cited by

• Eliminating Inductive Bias in Reward Models with Information-Theoretic Guidance
• Neurosymbolic Inference On Foundation Models For Remote Sensing Text-to-image Retrieval With Complex Queries
• MIDUS: Memory-Infused Depth Up-Scaling
• SATGround: A Spatially-Aware Approach for Visual Grounding in Remote Sensing
• Parameter efficient hybrid spiking-quantum convolutional neural network with surrogate gradient and quantum data-reupload
• Enhancing Instruction-Following Capabilities in Seq2Seq Models: DoLA Adaptations for T5
• Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understanding
• CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
• Towards Active Synthetic Data Generation for Finetuning Language Models
• ParaBlock: Communication-Computation Parallel Block Coordinate Federated Learning for Large Language Models
• ChainV: Atomic Visual Hints Make Multimodal Reasoning Shorter and Better
• Black-Box On-Policy Distillation of Large Language Models
• Advancing Subsurface Discovery and Geothermal Monitoring with an Agentic Artificial Intelligence Framework
• Personalized Decision Modeling: Utility Optimization or Textualized-Symbolic Reasoning
• SoK: Taxonomy and Evaluation of Prompt Security in Large Language Models
• Computational Budget Should Be Considered in Data Selection
• Cautious Weight Decay
• Learning Dynamics of VLM Finetuning
• MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
• You only need 4 extra tokens: Synergistic Test-time Adaptation for LLMs
• Fewer Weights, More Problems: A Practical Attack on LLM Pruning
• TeachLM: Post-Training LLMs for Education Using Authentic Learning Data
• Train on Validation (ToV): Fast data selection with applications to fine-tuning
• Judging with Confidence: Calibrating Autoraters to Preference Distributions
• VLM-FO1: Bridging the Gap Between High-Level Reasoning and Fine-Grained Perception in VLMs
• CORRECT: COndensed eRror RECognition via knowledge Transfer in multi-agent systems
• PG-CE: A Progressive Generation Dataset with Constraint Enhancement for Controllable Text Generation
• Memory-Efficient Federated Fine-Tuning of Large Language Models via Layer Pruning
• Pluralistic Off-policy Evaluation and Alignment
• Teaching LLMs to Plan: Logical Chain-of-Thought Instruction Tuning for Symbolic Planning
• AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
• A Survey of the State-of-the-Art in Conversational Question Answering Systems
• Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?
• Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient
• Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models
• Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning
• Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models
• Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challenges
• Learning Like Humans: Resource-Efficient Federated Fine-Tuning through Cognitive Developmental Stages
• LLMControl: Grounded Control of Text-to-Image Diffusion-based Synthesis with Multimodal LLMs

Discussions

• Instruction Tuning with GPT-4 (From Microsoft) [hn, 4 points, 0 comments]
• Instruction Tuning with GPT-4 [hn, 1 points, 0 comments]
• Instruction Tuning with GPT-4 [hn, 1 points, 0 comments]

Related

• Non-instructional Fine-tuning: Enabling Instruction-Following Capabilities in Pre-trained Language Models without Instruction-Following Data
• Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models
• Instruction Mining: Instruction Data Selection for Tuning Large Language Models
• Instruction Tuning for Large Language Models: A Survey
• Scaling Instruction-Finetuned Language Models