Éric de la Clergerie

1. Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck
   2024/04/11 by Nathan Godey, Éric de la Clergerie, Éric Villemonte de la Clergerie +4 · 2 voices · 6 citations
   Computer Science · #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL
2. Anisotropy Is Inherent to Self-Attention in Transformers
   2024/01/22 by Nathan Godey, Eric Villemonte de La Clergerie, Éric de la Clergerie +4 · 2 voices · 8 citations
   Computer Science · #cs.CL
3. Headless Language Models: Learning without Predicting with Contrastive Weight Tying
   2023/09/15 by Nathan Godey, Godey, Nathan, Éric de la Clergerie +4 · 4 voices · 1 citation
   Computer Science · #Topic Modeling #Natural Language Processing Techniques #Text Readability and Simplification
4. Gaperon: A Peppered English-French Generative Language Model Suite
   2025/10/29 by Nathan Godey, Wissam Antoun, Godey, Nathan +11 · 4 voices
   #cs.CL #cs.AI
5. OntoBook: Ontology-Grounded Synthetic Textbooks for Medical Encoder Pretraining
   2026/07/21 by Rian Touchent, Éric de la Clergerie
   #cs.AI