TULIP: Adapting Open-Source Large Language Models for Underrepresented Languages and Specialized Financial Tasks

2025/08/22 by Irem Nur Demirtas, Demirtaş, İrem, Burak Payzun +3
Computer Science · #Computation and Language (cs.CL) #Computational Physics and Python Applications #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2508.16243

openalex publication_date 2025/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Thanks to the growing popularity of large language models over the years, there is great potential for their applications in finance. Despite the exceptional performance of larger proprietary models, which are presented as black-box solutions through APIs, smaller models that can be hosted on-premise present opportunities for adaptability and privacy. Especially in cases where the management of sensitive information and application of domain knowledge is important, like finance, enhancing the capabilities of smaller models becomes crucial, notably for underrepresented languages. In this work, we introduce TULIP models, which adapt Llama 3.1 8B and Qwen 2.5 7B for domain and language adaptation, focusing on financial Turkish use cases. The five-stage development pipeline involves data collection, continual pre-training (CPT), benchmark design, synthetic data generation and supervised fine-tuning (SFT). The results show that the capabilities of the models can be enhanced to effectively accomplish targeted tasks in this specific domain and language.

Citations

• Capabilities of Gemini Models in Medicine
• FinBen: A Holistic Financial Benchmark for Large Language Models
• FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models
• Adapting Large Language Models to Domains via Reading Comprehension
• Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models
• FinGPT: Open-Source Financial Large Language Models
• PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance
• BloombergGPT: A Large Language Model for Finance
• WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain
• LoRA: Low-Rank Adaptation of Large Language Models

Related

• FinGPT: Open-Source Financial Large Language Models
• FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets
• The LLM Pro Finance Suite: Multilingual Large Language Models for Financial Applications
• Large Language Models in Finance: A Survey
• Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models