2024/03/20 by Shuaijiang Zhao, Zhao, Shuaijiang, Xiaoquan Fang +1
Computer Science · Decision Sciences · #Computation and Language (cs.CL) #Data Quality and Management #FOS: Computer and information sciences #Machine Learning and Data Classification #Natural Language Processing Techniques
paper · pdf · doi:10.48550/arxiv.2403.13233
openalex publication_date 2024/03/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In the era of flourishing large-scale models, the challenge of selecting and optimizing datasets from the vast and complex sea of data, to enhance the performance of large language models within the constraints of limited computational resources, has become paramount. This paper details our solution for the BetterMixture challenge, which focuses on the fine-tuning data mixing for large language models. Our approach, which secured third place, incorporates data deduplication, low-level and high-level quality filtering, and diversity selection. The foundation of our solution is Ke-Data-Juicer, an extension of Data-Juicer, demonstrating its robust capabilities in handling and optimizing data for large language models.