Adam Roberts

1. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
   2019/10/23 by Colin Raffel, Noam Shazeer, Raffel, Colin +16 · 4 voices · 1797 citations
   Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.LG #stat.ML
2. Extracting Training Data from Large Language Models
   2020/12/14 by Nicholas Carlini, Florian Tramèr, Carlini, Nicholas +25 · 8 voices · 364 citations
   Computer Science · #Adversarial Robustness in Machine Learning #Privacy-Preserving Technologies in Data #Topic Modeling #cs.CL #cs.CR #cs.LG
3. MuChoMusic dataset
   2023/01/26 by Andrea Agostinelli, Agostinelli, Andrea, Timo I. Denk +23 · 1 voice · 105 citations
   Computer Science · #Music Technology and Sound Studies #Music and Audio Processing #Speech Recognition and Synthesis #cs.LG #cs.SD #eess.AS
4. Large Language Models Struggle to Learn Long-Tail Knowledge
   2022/11/15 by Nikhil Kandpal, Haikang Deng, Kandpal, Nikhil +7 · 2 voices · 81 citations
   Computer Science · #Topic Modeling #Natural Language Processing Techniques #Expert finding and Q&A systems
5. Scaling Instruction-Finetuned Language Models
   2022/10/20 by Hyung Won Chung, Chung, Hyung Won, Le Hou +72 · 1 voice · 358 citations
   Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling #cs.CL #cs.LG
6. PaLM: Scaling Language Modeling with Pathways
   2022/04/05 by Aakanksha Chowdhery, Sharan Narang, Chowdhery, Aakanksha +129 · 589 citations
   Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
7. LaMDA: Language Models for Dialog Applications
   2022/01/20 by Romal Thoppilan, Daniel De Freitas, Thoppilan, Romal +126 · 2 voices · 69 citations
   Computer Science · #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling #cs.AI #cs.CL
8. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
   2022/11/09 by BigScience Workshop, Workshop, BigScience, Teven Le Scao +784 · 1 voice · 151 citations
   Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.CL
9. Gemma: Open Models Based on Gemini Research and Technology
   2024/03/13 by Thomas Mésnard, Gemma Team, Cassidy Hardin +203 · 214 citations
   Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multi-Agent Systems and Negotiation
10. Training LLMs over Neurally Compressed Text
    2024/04/04 by Brian Lester, Jaehoon Lee, Lester, Brian +12 · 4 voices · 4 citations
    Computer Science · #Handwritten Text Recognition Techniques #Mathematics, Computing, and Information Processing #Natural Language Processing Techniques #cs.CL #cs.LG
11. ByT5: Towards a token-free future with pre-trained byte-to-byte models
    2021/05/28 by Linting Xue, Aditya Barua, Xue, Linting +14 · 1 voice · 73 citations
    Computer Science · #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL
12. The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
    2023/01/31 by Shayne Longpre, Longpre, Shayne, Le Hou +18 · 99 citations
    Computer Science · Engineering · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Machine Learning and Data Classification
13. DDSP: Differentiable Digital Signal Processing
    2020/01/14 by Jesse Engel, Lamtharn Hantrakul, Engel, Jesse +6 · 1 voice · 43 citations
    Computer Science · #Music Technology and Sound Studies #Music and Audio Processing #Speech and Audio Processing #cs.LG #cs.SD #eess.AS #eess.SP #stat.ML
14. UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining
    2023/04/18 by Hyung Won Chung, Noah Constant, Chung, Hyung Won +11 · 40 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling
15. A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity
    2023/05/22 by Shayne Longpre, Gregory Yauney, Longpre, Shayne +18 · 38 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Software Engineering Research
16. Character-Aware Models Improve Visual Text Rendering
    2022/12/20 by Rosanne Liu, Liu, Rosanne, Dan Garrette +18 · 1 voice · 10 citations
    Computer Science · Engineering · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Human Motion and Animation #Multimodal Machine Learning Applications #cs.CL #cs.CV
17. What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?
    2022/04/12 by Thomas J. Wang, Adam Roberts, Wang, Thomas +13 · 19 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques #Topic Modeling
18. SingSong: Generating musical accompaniments from singing
    2023/01/30 by Chris Donahue, Donahue, Chris, Antoine Caillon +19 · 13 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Multimedia (cs.MM) #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering
19. Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models
    2017/11/15 by Jesse Engel, Engel, Jesse, Matthew Hoffman +5 · 7 citations
    Computer Science · Physics and Astronomy · #Computer Graphics and Visualization Techniques #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Human Pose and Action Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks #Neural and Evolutionary Computing (cs.NE)
20. VeLO: Training Versatile Learned Optimizers by Scaling Up
    2022/11/17 by Luke Metz, Metz, Luke, J. Harrison +22 · 2 voices · 6 citations
    Computer Science · #Advanced Neural Network Applications #Human Pose and Action Recognition #Machine Learning and Data Classification #cs.LG #math.OC #stat.ML
21. Report of the 1st Workshop on Generative AI and Law
    2023/11/11 by A. Feder Cooper, Katherine Lee, Cooper, A. Feder +67 · 4 citations
    Computer Science · Social Sciences · #Artificial Intelligence in Law #Computers and Society (cs.CY) #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Law, AI, and Intellectual Property