AMALIA Technical Report: A Fully Open Source Large Language Model for European Portuguese

2026/03/27 by Afonso Simplício, Goncalo Vinagre, Gonçalo Vinagre +22 · 1 voice
Computer Science · #Benchmarking #European Portuguese #Language model #Natural Language Processing Techniques #Open data #Open source #Portuguese #Suite #Text Readability and Simplification #Topic Modeling #cs.AI #cs.CL #cs.LG

paper · pdf · open access · doi:10.48550/arxiv.2603.26511

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2026/03/27 · arxiv published 2026/03/27 · arxiv updated 2026/03/27 · openalex created_date 2026/03/31 · openalex updated_date 2026/07/28

Abstract

Despite rapid progress in open large language models (LLMs), European Portuguese (pt-PT) remains underrepresented in both training data and native evaluation, with machine-translated benchmarks likely missing the variant's linguistic and cultural nuances. We introduce AMALIA, a fully open LLM that prioritizes pt-PT by using more high-quality pt-PT data during both the mid- and post-training stages. To evaluate pt-PT more faithfully, we release a suite of pt-PT benchmarks that includes translated standard tasks and four new datasets targeting pt-PT generation, linguistic competence, and pt-PT/pt-BR bias. Experiments show that AMALIA matches strong baselines on translated benchmarks while substantially improving performance on pt-PT-specific evaluations, supporting the case for targeted training and native benchmarking for European Portuguese.

Citations

• Qwen3Guard Technical Report
• NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
• The Bitter Lesson Learned from 2,000+ Multilingual Benchmarks
• MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
• Scaling Synthetic Data Creation with 1,000,000,000 Personas
• Advancing Generative AI for Portuguese with Open Decoder Gervásio PT*
• GlórIA -- A Generative and Open Large Language Model for Portuguese
• Effective Long-Context Scaling of Foundation Models
• Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• OpenAssistant Conversations -- Democratizing Large Language Model Alignment
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• Training Verifiers to Solve Math Word Problems
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• Measuring Massive Multitask Language Understanding
• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• Apertus: Democratizing Open and Compliant LLMs for Global Language Environments

Discussions

• 🧐 AMALIA Technical Report: A Fully Open Source #LLM for European Portuguese: https://arxiv.org/abs/2603.26511 [bsky, 1 points, 0 comments]

Related

• GlórIA -- A Generative and Open Large Language Model for Portuguese
• PoETa v2: Toward More Robust Evaluation of Large Language Models in Portuguese
• Tradutor: Building a Variety Specific Translation Model
• Building High-Quality Datasets for Portuguese LLMs: From Common Crawl Snapshots to Industrial-Grade Corpora
• PeLLE: Encoder-based language models for Brazilian Portuguese based on open data