MathScale: Scaling Instruction Tuning for Mathematical Reasoning

2024/03/05 by Zhengyang Tang, Tang, Zhengyang, Xingxing Zhang +5 · 64 citations
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Machine Learning (cs.LG) #Mathematics Education and Teaching Techniques

paper · pdf · doi:10.48550/arxiv.2403.02884

openalex publication_date 2024/03/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities in problem-solving. However, their proficiency in solving mathematical problems remains inadequate. We propose MathScale, a simple and scalable method to create high-quality mathematical reasoning data using frontier LLMs (e.g., \tt GPT-3.5). Inspired by the cognitive mechanism in human mathematical learning, it first extracts topics and knowledge points from seed math questions and then build a concept graph, which is subsequently used to generate new math questions. MathScale exhibits effective scalability along the size axis of the math dataset that we generate. As a result, we create a mathematical reasoning dataset (MathScaleQA) containing two million math question-answer pairs. To evaluate mathematical reasoning abilities of LLMs comprehensively, we construct \sc MwpBench, a benchmark of Math Word Problems, which is a collection of ten datasets (including GSM8K and MATH) covering K-12, college, and competition level math problems. We apply MathScaleQA to fine-tune open-source LLMs (e.g., LLaMA-2 and Mistral), resulting in significantly improved capabilities in mathematical reasoning. Evaluated on \sc MwpBench, MathScale-7B achieves state-of-the-art performance across all datasets, surpassing its best peers of equivalent size by 42.9% in micro average accuracy and 43.7% in macro average accuracy, respectively.

Cited by

• Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data
• PS-PPO: Prefix-Sampling PPO for Critic-Free RLHF
• Accelerate Speculative Decoding with Sparse Computation in Verification
• Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Models
• From Imitation to Discrimination: Toward A Generalized Curriculum Advantage Mechanism Enhancing Cross-Domain Reasoning Tasks
• HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs
• Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis for Large Reasoning Models
• Do LLMs Signal When They're Right? Evidence from Neuron Agreement
• Count Counts: Motivating Exploration in LLM Reasoning with Count-based Intrinsic Rewards
• AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generation
• QueST: Incentivizing LLMs to Generate Difficult Problems
• GroundedPRM: Tree-Guided and Fidelity-Aware Process Reward Modeling for Step-Level Reasoning
• Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
• EDUMATH: Generating Standards-aligned Educational Math Word Problems
• Expanding the Action Space of LLMs to Reason Beyond Language
• Making Mathematical Reasoning Adaptive
• LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning
• Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning
• Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs
• ZeroSiam: An Efficient Siamese for Test-Time Entropy Optimization without Collapse
• ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoning
• PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning
• Σ-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systems
• CogAtom: From Cognitive Atoms to Olympiad-level Mathematical Reasoning in Large Language Models
• SCAN: Self-Denoising Monte Carlo Annotation for Robust Process Reward Learning
• The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward
• SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
• Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction
• MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy
• WarriorMath: Enhancing the Mathematical Ability of Large Language Models with a Defect-aware Framework
• WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
• A Survey of Deep Learning for Geometry Problem Solving
• SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
• A Survey on Large Language Models for Mathematical Reasoning
• Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
• Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTS
• BlueLM-2.5-3B Technical Report
• Does Learning Mathematical Problem-Solving Generalize to Broader Reasoning?
• EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
• AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking
• Synthesis by Design: Controlled Data Generation via Structural Guidance
• SciDA: Scientific Dynamic Assessor of LLMs
• SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation
• Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification
• MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning
• Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scaling
• Diversity-Aware Policy Optimization for Large Language Model Reasoning
• Self-Error-Instruct: Generalizing from Errors for LLMs Mathematical Reasoning
• Beyond Templates: Dynamic Adaptation of Reasoning Demonstrations via Feasibility-Aware Exploration
• Beyond Markovian: Reflective Exploration via Bayes-Adaptive RL for LLM Reasoning
• HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation
• SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data
• GeoGramBench: Benchmarking the Geometric Program Reasoning in Modern LLMs
• Dynamic Sampling that Adapts: Iterative DPO for Self-Aware Mathematical Reasoning
• Scaling Physical Reasoning with the PHYSICS Dataset
• TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
• Activation-Guided Consensus Merging for Large Language Models
• SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation
• Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization
• DEL: Digit Entropy Loss for Numerical Learning of Large Language Models
• RV-Syn: Rational and Verifiable Mathematical Reasoning Data Synthesis based on Structured Function Library
• Efficient Reasoning for LLMs through Speculative Chain-of-Thought

Related

• MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs
• Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On
• Common 7B Language Models Already Possess Strong Math Capabilities
• MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contexts
• Augmenting Math Word Problems via Iterative Question Composing