Smith, Shaden

1. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
   2022/11/09 by Workshop, BigScience, :, Scao, Teven Le +391 · 110 citations
   #Computation and Language (cs.CL) #FOS: Computer and information sciences
2. DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
   2022/06/30 by Reza Yazdani Aminabadi, Samyam Rajbhandari, Aminabadi, Reza Yazdani +19 · 60 citations
   Computer Science · Engineering · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Graph Theory and Algorithms #Machine Learning (cs.LG) #Parallel #Performance (cs.PF) #and Cluster Computing (cs.DC)
3. ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
   2021/04/16 by Samyam Rajbhandari, Olatunji Ruwase, Rajbhandari, Samyam +7 · 42 citations
   Computer Science · #Advanced Neural Network Applications #Stochastic Gradient Optimization Techniques #Parallel Computing and Optimization Techniques
4. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
   2022/01/28 by Shaden Smith, Smith, Shaden, Mostofa Patwary +36 · 26 citations
   Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling