2024/05/06 by Bin Wang, Geyu Lin, Wang, Bin +7 · 1 citation
Arts and Humanities · Computer Science · #Computation and Language (cs.CL) #Diverse Musicological Studies #FOS: Computer and information sciences #Music and Audio Processing
paper · pdf · doi:10.48550/arxiv.2405.03138
openalex publication_date 2024/05/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Large language models (LLMs) have rapidly evolved as the foundation of various natural language processing (NLP) applications. Despite their wide use cases, their understanding of culturally-related concepts and reasoning remains limited. Meantime, there is a significant need to enhance these models' cultural reasoning capabilities, especially concerning underrepresented regions. This paper introduces a novel pipeline for extracting high-quality, culturally-related instruction tuning datasets from vast unstructured corpora. We utilize a self-instruction generation pipeline to identify cultural concepts and trigger instruction. By integrating with a general-purpose instruction tuning dataset, our model demonstrates enhanced capabilities in recognizing and understanding regional cultural nuances, thereby enhancing its reasoning capabilities. We conduct experiments across three regions: Singapore, the Philippines, and the United States, achieving performance improvement of up to 6%. Our research opens new avenues for extracting cultural instruction tuning sets directly from unstructured data, setting a precedent for future innovations in the field.