Vijay Anand Korthikanti

1. An Empirical Study of Mamba-based Language Models
   2024/06/12 by Roger Waleffe, Waleffe, Roger, Wonmin Byeon +30 · 1 voice · 54 citations
   Arts and Humanities · #Language, Linguistics, Cultural Analysis #cs.CL #cs.LG
2. Efficient Large-Scale Language Model Training on GPU Clusters Using\n Megatron-LM
   2021/04/09 by Deepak Narayanan, Mohammad Shoeybi, Narayanan, Deepak +21 · 125 citations
   Computer Science · #Computation and Language (cs.CL) #Distributed #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Parallel #Topic Modeling #and Cluster Computing (cs.DC)
3. Reducing Activation Recomputation in Large Transformer Models
   2022/05/10 by Vijay Anand Korthikanti, Jared Casper, Korthikanti, Vijay +11 · 65 citations
   Computer Science · #Advanced Neural Network Applications #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel Computing and Optimization Techniques #Topic Modeling
4. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
   2022/01/28 by Shaden Smith, Smith, Shaden, Mostofa Patwary +36 · 31 citations
   Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
5. Upcycling Large Language Models into Mixture of Experts
   2024/10/10 by Ethan He, Abhinav Khattar, He, Ethan +17 · 16 citations
   Computer Science · #Expert finding and Q&A systems #Topic Modeling
6. Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning
   2023/02/09 by Zhuolin Yang, Yang, Zhuolin, Ping, Wei +28 · 6 citations
   Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
7. Scalable Training of Mixture-of-Experts Models with Megatron Core
   2026/03/08 by Zijie Yan, Hongxiao Bai, Xin Yao +42 · 3 voices · 2 citations
   #cs.DC #cs.CL #cs.LG