Krikri: Advancing Open Large Language Models for Greek

2025/05/19 by Dimitris Roussis, Roussis, Dimitris, Leon Voukoutis +15 · 3 citations
Computer Science · #Authorship Attribution and Profiling #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2505.13772

openalex publication_date 2025/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce Llama-Krikri-8B, a cutting-edge Large Language Model tailored for the Greek language, built on Meta's Llama 3.1-8B. Llama-Krikri-8B has been extensively trained on high-quality Greek data to ensure superior adaptation to linguistic nuances. With 8 billion parameters, it offers advanced capabilities while maintaining efficient computational performance. Llama-Krikri-8B supports both Modern Greek and English, and is also equipped to handle polytonic text and Ancient Greek. The chat version of Llama-Krikri-8B features a multi-stage post-training pipeline, utilizing both human and synthetic instruction and preference data, by applying techniques such as MAGPIE. In addition, for evaluation, we propose three novel public benchmarks for Greek. Our evaluation on existing as well as the proposed benchmarks shows notable improvements over comparable Greek and multilingual LLMs in both natural language understanding and generation as well as code generation.

Citations

• SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
• Preference Leakage: A Contamination Problem in LLM-as-a-judge
• BgGPT 1.0: Extending English-centric LLMs to other languages
• Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier
• Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translation
• Bielik 7B v0.1: A Polish Language Model -- Development, Insights, and Evaluation
• Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs
• Liger Kernel: Efficient Triton Kernels for LLM Training
• HelpSteer2-Preference: Complementing Ratings with Preferences
• EuroLLM: Multilingual Language Models for Europe
• Meltemi: The first open Large Language Model for Greek
• AgentInstruct: Toward Generative Teaching with Agentic Flows
• RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs
• WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
• The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
• From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline
• Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
• HelpSteer2: Open-source dataset for training top-performing reward models
• Advancing LLM Reasoning Generalists with Preference Trees
• Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
• How do Large Language Models Handle Multilingualism?
• StarCoder 2 and The Stack v2: The Next Generation
• Tower: An Open Multilingual Large Language Model for Translation-Related Tasks
• A Survey on Data Selection for Language Models
• Efficiently Adapting Pretrained Language Models To New Languages
• Qwen Technical Report
• CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages
• The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants
• An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Affective Coherence Monitoring for Transformer-Based Language Models
• CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task
• Training language models to follow instructions with human feedback
• Multi-granular Legal Topic Classification on Greek Legislation
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer
• Deduplicating Training Data Makes Language Models Better
• Measuring Massive Multitask Language Understanding
• Language Models are Few-Shot Learners
• Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
• Large-Scale Multi-Label Text Classification on EU Legislation
• HellaSwag: Can a Machine Really Finish Your Sentence?
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?
• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• Decoupled Weight Decay Regularization
• Bag of Tricks for Efficient Text Classification
• Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts
• UltraFeedback: Boosting Language Models with Scaled AI Feedback
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• GRDD+: An Extended Greek Dialectal Dataset with Cross-Architecture Fine-tuning Evaluation
• VOX-KRIKRI: Unifying Speech and Language through Continuous Fusion
• GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citations

Related

• Meltemi: The first open Large Language Model for Greek
• GreekBART: The First Pretrained Greek Sequence-to-Sequence Model
• Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation
• Open or Closed LLM for Lesser-Resourced Languages? Lessons from Greek
• GR-NLP-TOOLKIT: An Open-Source NLP Toolkit for Modern Greek