Latin BERT: A Contextual Language Model for Classical Philology

2020/09/21 by David Bamman, Bamman, David, Patrick J. Burns +1 · 4 citations
Computer Science · Arts and Humanities · #Natural Language Processing Techniques #Topic Modeling #Digital Humanities and Scholarship

paper · pdf · doi:10.48550/arxiv.2009.10053

Abstract

We present Latin BERT, a contextual language model for the Latin language, trained on 642.7 million words from a variety of sources spanning the Classical era to the 21st century. In a series of case studies, we illustrate the affordances of this language-specific model both for work in natural language processing for Latin and in using computational methods for traditional scholarship: we show that Latin BERT achieves a new state of the art for part-of-speech tagging on all three Universal Dependency datasets for Latin and can be used for predicting missing text (including critical emendations); we create a new dataset for assessing word sense disambiguation for Latin and demonstrate that Latin BERT outperforms static word embeddings; and we show that it can be used for semantically-informed search by querying contextual nearest neighbors. We publicly release trained models to help drive future work in this space.

Cited by

• Contextual morphologically-guided tokenization for Latin encoder models
• Contextualizing ancient texts with generative neural networks
• Reviving Cultural Heritage: A Novel Approach for Comprehensive Historical Document Restoration
• Latin writing styles analysis with Machine Learning: New approach to old questions

Related

• Sentence Embedding Models for Ancient Greek Using Multilingual Knowledge Distillation
• LiMe: a Latin Corpus of Late Medieval Criminal Sentences
• Detecting Latin in Historical Books with Large Language Models: A Multimodal Benchmark
• Exploring Large Language Models for Classical Philology
• "Don't Teach Minerva": Guiding LLMs Through Complex Syntax for Faithful Latin Translation with RAG