Jeff Rasley

1. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
   2019/10/04 by Samyam Rajbhandari, Jeff Rasley, Rajbhandari, Samyam +5 · 267 citations
   Computer Science · Engineering · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel #Topic Modeling #and Cluster Computing (cs.DC)
2. DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
   2022/06/30 by Reza Yazdani Aminabadi, Aminabadi, Reza Yazdani, Samyam Rajbhandari +19 · 72 citations
   Computer Science · Engineering · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Graph Theory and Algorithms #Machine Learning (cs.LG) #Parallel #Performance (cs.PF) #and Cluster Computing (cs.DC)
3. ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
   2021/04/16 by Samyam Rajbhandari, Olatunji Ruwase, Rajbhandari, Samyam +7 · 49 citations
   Computer Science · #Advanced Neural Network Applications #Stochastic Gradient Optimization Techniques #Parallel Computing and Optimization Techniques
4. DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
   2023/08/02 by Zhewei Yao, Yao, Zhewei, Reza Yazdani Aminabadi +35 · 16 citations
   Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification
5. DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
   2024/01/09 by Connor Holmes, Holmes, Connor, Masahiro Tanaka +19 · 19 citations
   Computer Science · Engineering · #Topic Modeling #Natural Language Processing Techniques #Ferroelectric and Negative Capacitance Devices
6. DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
   2023/10/06 by Shuaiwen Leon Song, Song, Shuaiwen Leon, Bonnie Kruft +181 · 4 citations
   Biochemistry, Genetics and Molecular Biology · Decision Sciences · Materials Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Genetics, Bioinformatics, and Biomedical Research #Machine Learning (cs.LG) #Machine Learning in Materials Science #Scientific Computing and Data Management