Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scale

2025/09/17 by Hasan Abed Al Kader Hammoud, Hammoud, Hasan Abed Al Kader, Mohammad Zbeeb +3 · 2 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Text Readability and Simplification

paper · pdf · doi:10.48550/arxiv.2509.14008

Abstract

We present Hala, a family of Arabic-centric instruction and translation models built with our translate-and-tune pipeline. We first compress a strong AR↔EN teacher to FP8 (yielding ∼2× higher throughput with no quality loss) and use it to create high-fidelity bilingual supervision. A lightweight language model LFM2-1.2B is then fine-tuned on this data and used to translate high-quality English instruction sets into Arabic, producing a million-scale corpus tailored to instruction following. We train Hala models at 350M, 700M, 1.2B, and 9B parameters, and apply slerp merging to balance Arabic specialization with base-model strengths. On Arabic-centric benchmarks, Hala achieves state-of-the-art results within both the "nano" (≤2B) and "small" (7-9B) categories, outperforming their bases. We release models, data, evaluation, and recipes to accelerate research in Arabic NLP.

Citations

• Kimi K2: Open Agentic Intelligence
• The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technology
• Qwen3 Technical Report
• Gemma 3 Technical Report
• Science Across Languages: Assessing LLM Multilingual Translation of Scientific Papers
• SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domain
• Fanar: An Arabic-Centric Multimodal Generative AI Platform
• Tulu 3: Pushing Frontiers in Open Language Model Post-Training
• Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs
• Hermes 3 Technical Report
• The Llama 3 Herd of Models
• ALLaM: Large Language Models for Arabic and English
• Arcee's MergeKit: A Toolkit for Merging Large Language Models
• A New Massive Multilingual Dataset for High-Performance Language Technologies
• AraTrust: An Evaluation of Trustworthiness for LLMs in Arabic
• Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks
• Reformatted Alignment
• The Falcon Series of Open Language Models
• AceGPT, Localizing Large Language Models in Arabic
• Baichuan 2: Open Large-scale Language Models
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Orca: Progressive Learning from Complex Explanation Traces of GPT-4
• Early Weight Averaging meets High Learning Rates for LLM Pre-training
• Exploring Human-Like Translation Strategy with Large Language Models
• A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models
• LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
• Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis
• GPT-4 Technical Report
• Self-Instruct: Aligning Language Models with Self-Generated Instructions
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
• FP8 Quantization: The Power of the Exponent
• MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
• EXAMS: A Multi-Subject High School Examinations Dataset for\n Cross-Lingual and Multilingual Question Answering
• mT5: A massively multilingual pre-trained text-to-text transformer
• The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT
• Language Models are Few-Shot Learners
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation
• AraBERT: Transformer-based Model for Arabic Language Understanding
• Unsupervised Cross-lingual Representation Learning at Scale

Cited by

• SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data
• AraLingBench A Human-Annotated Benchmark for Evaluating Arabic Linguistic Capabilities of Large Language Models

Related

• GemmAr: Enhancing LLMs Through Arabic Instruction-Tuning
• ALLaM: Large Language Models for Arabic and English
• Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic
• Beyond English: Evaluating LLMs for Arabic Grammatical Error Correction
• Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLM