2024/06/05 by Yichuan Wang, Wang, Yichuan, Yang Liu +8
Computer Science · Decision Sciences · Engineering · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Distributed and Parallel Computing Systems #FOS: Computer and information sciences #Robotics and Automated Systems #Simulation Techniques and Applications
paper · pdf · doi:10.48550/arxiv.2406.02856
openalex publication_date 2024/06/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce Xmodel-LM, a compact and efficient 1.1B language model pre-trained on around 2 trillion tokens. Trained on our self-built dataset (Xdata), which balances Chinese and English corpora based on downstream task optimization, Xmodel-LM exhibits remarkable performance despite its smaller size. It notably surpasses existing open-source language models of similar scale. Our model checkpoints and code are publicly accessible on GitHub at https://github.com/XiaoduoAILab/XmodelLM.