OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset

2024/02/15 by Shubham Toshniwal, Ivan Moshkov, Toshniwal, Shubham +9 · 26 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Machine Learning (cs.LG) #Mathematics, Computing, and Information Processing

paper · pdf · doi:10.48550/arxiv.2402.10176

openalex publication_date 2024/02/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent work has shown the immense potential of synthetically generated datasets for training large language models (LLMs), especially for acquiring targeted skills. Current large-scale math instruction tuning datasets such as MetaMathQA (Yu et al., 2024) and MAmmoTH (Yue et al., 2024) are constructed using outputs from closed-source LLMs with commercially restrictive licenses. A key reason limiting the use of open-source LLMs in these data generation pipelines has been the wide gap between the mathematical skills of the best closed-source LLMs, such as GPT-4, and the best open-source LLMs. Building on the recent progress in open-source LLMs, our proposed prompting novelty, and some brute-force scaling, we construct OpenMathInstruct-1, a math instruction tuning dataset with 1.8M problem-solution pairs. The dataset is constructed by synthesizing code-interpreter solutions for GSM8K and MATH, two popular math reasoning benchmarks, using the recently released and permissively licensed Mixtral model. Our best model, OpenMath-CodeLlama-70B, trained on a subset of OpenMathInstruct-1, achieves a score of 84.6% on GSM8K and 50.7% on MATH, which is competitive with the best gpt-distilled models. We release our code, models, and the OpenMathInstruct-1 dataset under a commercially permissive license.

Cited by

• Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervision
• Towards Active Synthetic Data Generation for Finetuning Language Models
• ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
• Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis for Large Reasoning Models
• Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO
• FLEX: Continuous Agent Evolution via Forward Learning from Experience
• TempoBench: Evaluating Temporal Causal Reasoning in Large Language Models
• PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs
• Chain of Execution Supervision Promotes General Reasoning in Large Language Models
• AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generation
• Midtraining Bridges Pretraining and Posttraining Distributions
• To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models
• HoneyBee: Data Recipes for Vision-Language Reasoners
• Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for Indian Languages
• EDUMATH: Generating Standards-aligned Educational Math Word Problems
• Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning
• Read Between the Lines: A Benchmark for Uncovering Political Bias in Bangla News Articles
• Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Training
• MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources
• ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoning
• PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactions
• Reverse-Engineered Reasoning for Open-Ended Generation
• FLAMES: Improving LLM Math Reasoning via a Fine-Grained Analysis of the Data Synthesis Pipeline
• Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks
• Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
• Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction

Related

• OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data
• MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning
• MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
• MegaMath: Pushing the Limits of Open Math Corpora
• MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models