Introducing DictaLM -- A Large Generative Language Model for Modern Hebrew

2023/09/25 by Shaltiel Shmidman, Shmidman, Shaltiel, Avi Shmidman +5 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2309.14568

openalex publication_date 2023/09/25 · openalex created_date 2023/09/28 · openalex updated_date 2026/07/28

Abstract

We present DictaLM, a large-scale language model tailored for Modern Hebrew. Boasting 7B parameters, this model is predominantly trained on Hebrew-centric data. As a commitment to promoting research and development in the Hebrew language, we release both the foundation model and the instruct-tuned model under a Creative Commons license. Concurrently, we introduce DictaLM-Rab, another foundation model geared towards Rabbinic/Historical Hebrew. These foundation models serve as ideal starting points for fine-tuning various Hebrew-specific tasks, such as instruction, Q&A, sentiment analysis, and more. This release represents a preliminary step, offering an initial Hebrew LLM model for the Hebrew NLP community to experiment with.

Cited by

• Same violence, different answer: how AI responds to coercive control against women across languages

Related

• Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities
• DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew
• Knesset-DictaBERT: A Hebrew Language Model for Parliamentary Proceedings
• AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With
• Large Pre-Trained Models with Extra-Large Vocabularies: A Contrastive Analysis of Hebrew BERT Models and a New One to Outperform Them All