Guilherme Penedo
- The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
2025/06/05 by Nikhil Kandpal, Brian Lester, Kandpal, Nikhil +51 · 20 voices · 9 citations
Computer Science · Medicine · #Topic Modeling #Hate Speech and Cyberbullying Detection #Artificial Intelligence in Healthcare and Education
- The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only
2023/06/01 by Guilherme Penedo, Penedo, Guilherme, Quentin Malartic +16 · 2 voices · 55 citations
Computer Science · #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling #cs.AI #cs.CL
- The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
2024/06/25 by Guilherme Penedo, Hynek Kydlíček, Penedo, Guilherme +13 · 2 voices · 206 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.CL
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
2025/02/04 by Loubna Ben Allal, Anton Lozhkov, Allal, Loubna Ben +41 · 5 voices · 71 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.CL
- FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
2025/06/26 by Guilherme Penedo, Hynek Kydlíček, Penedo, Guilherme +17 · 1 voice · 33 citations
Computer Science · #Computation and Language (cs.CL) #Educational Technology and Assessment #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Online Learning and Analytics