Reza Yazdani Aminabadi
- DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
2022/06/30 by Reza Yazdani Aminabadi, Samyam Rajbhandari, Aminabadi, Reza Yazdani +19 · 65 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Graph Theory and Algorithms #Machine Learning (cs.LG) #Parallel #Performance (cs.PF) #and Cluster Computing (cs.DC)
- ZeRO-Offload: Democratizing Billion-Scale Model Training
2021/01/18 by Jie Ren, Samyam Rajbhandari, Ren, Jie +13 · 55 citations
Computer Science · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #Parallel Computing and Optimization Techniques #Topic Modeling #and Cluster Computing (cs.DC)
- Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
2022/01/28 by Shaden Smith, Smith, Shaden, Mostofa Patwary +36 · 29 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
- DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
2023/08/02 by Zhewei Yao, Yao, Zhewei, Reza Yazdani Aminabadi +35 · 15 citations
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification
- DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
2024/01/09 by Connor Holmes, Holmes, Connor, Masahiro Tanaka +19 · 17 citations
Computer Science · Engineering · #Topic Modeling #Natural Language Processing Techniques #Ferroelectric and Negative Capacitance Devices