Orca: Progressive Learning from Complex Explanation Traces of GPT-4

2023/06/05 by Subhabrata Mukherjee, Mukherjee, Subhabrata, Arindam Mitra +10 · 12 voices · 71 citations
Computer Science · Psychology · #Artificial intelligence #Benchmark (surveying) #Computer science #Explainable Artificial Intelligence (XAI) #Imitation #Machine Learning in Healthcare #Machine learning #Process (computing) #Psychology #Scale (ratio) #Social psychology #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2306.02707

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2023/06/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Recent research has focused on enhancing the capability of smaller models through imitation learning, drawing on the outputs generated by large foundation models (LFMs). A number of issues impact the quality of these models, ranging from limited imitation signals from shallow LFM outputs; small scale homogeneous training data; and most notably a lack of rigorous evaluation resulting in overestimating the small model's capability as they tend to learn to imitate the style, but not the reasoning process of LFMs. To address these challenges, we develop Orca (We are working with our legal team to publicly release a diff of the model weights in accordance with LLaMA's release policy to be published at https://aka.ms/orca-lm), a 13-billion parameter model that learns to imitate the reasoning process of LFMs. Orca learns from rich signals from GPT-4 including explanation traces; step-by-step thought processes; and other complex instructions, guided by teacher assistance from ChatGPT. To promote this progressive learning, we tap into large-scale and diverse imitation data with judicious sampling and selection. Orca surpasses conventional state-of-the-art instruction-tuned models such as Vicuna-13B by more than 100% in complex zero-shot reasoning benchmarks like Big-Bench Hard (BBH) and 42% on AGIEval. Moreover, Orca reaches parity with ChatGPT on the BBH benchmark and shows competitive performance (4 pts gap with optimized system message) in professional and academic examinations like the SAT, LSAT, GRE, and GMAT, both in zero-shot settings without CoT; while trailing behind GPT-4. Our research indicates that learning from step-by-step explanations, whether these are generated by humans or more advanced AI models, is a promising direction to improve model capabilities and skills.

Cited by

• From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search
• Learning Obfuscations Of LLM Embedding Sequences: Stained Glass Transform
• Sentence Splitter: Uncovering Latent Factual Structure for Self-Supervised Learning
• Gamayun's Path to Multilingual Mastery: Cost-Efficient Training of a 1.5B-Parameter LLM
• Prompt Governance? On Governing Technologies Governed by Natural Language
• GLaD: Geometric Latent Distillation for Vision-Language-Action Models
• PCMind-2.1-Kaiyuan-2B Technical Report
• CryptoTensors: A Light-Weight Large Language Model File Format for Highly-Secure Model Distribution
• Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models
• A Unified Understanding of Offline Data Selection and Online Self-refining Generation for Post-training LLMs
• SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data
• Monte Carlo Expected Threat (MOCET) Scoring
• Tailored Primitive Initialization is the Secret Key to Reinforcement Learning
• On the Notion that Language Models Reason
• Structured Definitions and Segmentations for Legal Reasoning in LLMs: A Study on Indian Legal Data
• NOTAM-Evolve: A Knowledge-Guided Self-Evolving Optimization Framework with LLMs for NOTAM Interpretation
• Importance-Aware Data Selection for Efficient LLM Instruction Tuning
• Autodata: An agentic data scientist to create high quality synthetic data
• BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
• Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hacking
• Beyond Imitation: Recovering Dense Rewards from Demonstrations
• Can RL Improve Generalization of LLM Agents? An Empirical Study
• Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation
• Decoupling Safety into Orthogonal Subspace: Cost-Efficient and Performance-Preserving Alignment for Large Language Models
• OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment
• Uncovering the Computational Ingredients of Human-Like Representations in LLMs
• Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios
• COSPADI: Compressing LLMs via Calibration-Guided Sparse Dictionary Learning
• Cross-Modal Knowledge Distillation for Speech Large Language Models
• Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scale
• A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
• MedQARo: A Large-Scale Benchmark for Medical Question Answering in Romanian
• Make me an Expert: Distilling from Generalist Black-Box Models into Specialized Models for Semantic Segmentation
• Fin-PRM: A Domain-Specialized Process Reward Model for Financial Reasoning in Large Language Models
• RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns
• Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models
• EvolvR: Self-Evolving Pairwise Reasoning for Story Evaluation to Enhance Generation
• Modelling and Classifying the Components of a Literature Review
• DialogueForge: LLM Simulation of Human-Chatbot Dialogue
• Decomposer: Learning to Decompile Symbolic Music to Programs
• Adaptive Repetition for Mitigating Position Bias in LLM-Based Ranking
• Replacing thinking with tool usage enables reasoning in small language models
• TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection
• Stylometry recognizes human and LLM-generated texts in short samples
• KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model
• A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools
• AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes
• Bridging the Digital Divide: Small Language Models as a Pathway for Physics and Photonics Education in Underdeveloped Regions
• SIPDO: Closed-Loop Prompt Optimization via Synthetic Data Feedback
• Online Knowledge Distillation with Reward Guidance
• Knowledge Grafting of Large Language Models
• The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
• Distilling LLM Agent into Small Models with Retrieval and Code Tools
• BeamClean: Language Aware Embedding Reconstruction
• A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
• DataClaw0: Agentic Tailoring Multimodal Data from Raw Streams
• Large Language Models for Computer-Aided Design: A Survey
• Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models
• Phi-4-reasoning Technical Report
• SkillGen: Verified Inference-Time Agent Skill Synthesis
• ToolTok: Tool Tokenization for Efficient and Generalizable GUI Agents
• The Frontier LLM Trap in Network Automation
• Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability
• BELL: Benchmarking the Explainability of Large Language Models
• Improving In-Context Learning with Reasoning Distillation
• A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook

Discussions

• Orca: Progressive Learning from Complex Explanation Traces of GPT-4 [hn, 113 points, 29 comments]
• Orca: Progressive Learning from Complex Explanation Traces of GPT-4 [hn, 11 points, 3 comments]
• Orca: The Model Few Saw Coming [lemmy, 8 points, 2 comments]
• Orca: Progressive Learning from Complex Explanation Traces of GPT-4 [hn, 6 points, 0 comments]
• Orca: Progressive Learning from Complex Explanation Traces of GPT-4 [hn, 4 points, 1 comments]
• Orca - Progressive Learning from Complex Explanation Traces of GPT-4 [lemmy, 3 points, 0 comments]
• Orca: Progressive Learning from Complex Explanation Traces of GPT-4 [hn, 3 points, 0 comments]
• Orca: Progressive Learning from Complex Explanation Traces of GPT-4 [hn, 2 points, 0 comments]
• Missed this - thanks for the recommendation https://arxiv.org/abs/2306.02707 [bsky, 2 points, 1 comments]
• We have reached the AI-teaching-AI stage of the journey. This is really cool - Orca is 13B parameters, so you can run this locally (i.e. there are probably pocket-sized devices that can have this nat [bsky, 1 points, 1 comments]
• https://arxiv.org/abs/2306.02707 [bsky, 0 points, 0 comments]
• Microsoft releases a research paper about an extremely capable AI/LLM called “Orca”… and at the same time orca’s start attacking and sinking boats in the Mediterranean. Coincidence? You be the judge. [bsky, 0 points, 0 comments]

Related

• Orca 2: Teaching Small Language Models How to Reason
• The ORCA Benchmark: Evaluating Real-World Calculation Accuracy in Large Language Models
• Orca-Math: Unlocking the potential of SLMs in Grade School Math
• Learning to Reason: Training LLMs with GPT-OSS or DeepSeek R1 Reasoning Traces
• GPT-3.5, GPT-4, or BARD? Evaluating LLMs Reasoning Ability in Zero-Shot Setting and Performance Boosting Through Prompts