Matteo Pagliardini
- DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging
2024/02/04 by Matteo Pagliardini, Amirkeivan Mohtashami, Pagliardini, Matteo +6 · 1 voice · 13 citations
Computer Science · #Neural Networks and Applications
- MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
2023/11/27 by Zeming Chen, Chen, Zeming, A. Cano +37 · 44 citations
Computer Science · Medicine · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Topic Modeling
- DoGE: Domain Reweighting with Generalization Estimation
2023/10/23 by Simin Fan, Matteo Pagliardini, Fan, Simin +3 · 14 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling
- The AdEMAMix Optimizer: Better, Faster, Older
2024/09/05 by Matteo Pagliardini, Pierre Ablin, Pagliardini, Matteo +3 · 10 citations
Engineering · Medicine · #FOS: Computer and information sciences #Fault Detection and Control Systems #Hemodynamic Monitoring and Therapy #Machine Learning (cs.LG) #Machine Learning (stat.ML)
- Benchmarking Optimizers for Large Language Model Pretraining
2025/09/01 by A L Semenov, Andrei Semenov, Matteo Pagliardini +4 · 2 voices · 15 citations
Computer Science · Social Sciences · #Computational and Text Analysis Methods #Machine Learning and Data Classification #Topic Modeling #cs.LG
- Leveraging the true depth of LLMs
2025/02/05 by Ramón Calvo González, González, Ramón Calvo, Daniele Paliotta +7 · 2 voices · 1 citation
#cs.LG #cs.CL