SkyMath: Technical Report

2023/10/25 by Liu Yang, Haihua Yang, Yang, Liu +31
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Mathematics, Computing, and Information Processing

paper · pdf · doi:10.48550/arxiv.2310.16713

openalex publication_date 2023/10/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) have shown great potential to solve varieties of natural language processing (NLP) tasks, including mathematical reasoning. In this work, we present SkyMath, a large language model for mathematics with 13 billion parameters. By applying self-compare fine-tuning, we have enhanced mathematical reasoning abilities of Skywork-13B-Base remarkably. On GSM8K, SkyMath outperforms all known open-source models of similar size and has established a new SOTA performance.

Related

• Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On
• MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
• SKYLENAGE Technical Report: Mathematical Reasoning and Contest-Innovation Benchmarks for Multi-Level Math Evaluation
• Common 7B Language Models Already Possess Strong Math Capabilities
• Large Language Models for Mathematical Reasoning: Progresses and Challenges