vix.ing · top · new · best · stats · spec

Mohammad Shoeybi

  1. Deep Voice: Real-time Neural Text-to-Speech
    2017/02/25 by Sercan O. Arik, Sercan Ö. Arık, Mike Chrzanowski +24 · 2 voices · 29 citations
    Computer Science · #Speech Recognition and Synthesis #Natural Language Processing Techniques #Topic Modeling
  2. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
    2019/09/17 by Mohammad Shoeybi, Shoeybi, Mohammad, Mostofa Patwary +9 · 326 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Speech Recognition and Synthesis
  3. An Empirical Study of Mamba-based Language Models
    2024/06/12 by Roger Waleffe, Wonmin Byeon, Waleffe, Roger +30 · 1 voice · 46 citations
    Arts and Humanities · #Language, Linguistics, Cultural Analysis #cs.CL #cs.LG
  4. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
    2024/05/27 by Chankyu Lee, Rajarshi Roy, Lee, Chankyu +12 · 2 voices · 111 citations
    Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Law #Computation and Language (cs.CL) #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Natural Language Processing Techniques #cs.AI #cs.CL #cs.IR #cs.LG
  5. Efficient Large-Scale Language Model Training on GPU Clusters Using\n Megatron-LM
    2021/04/09 by Deepak Narayanan, Narayanan, Deepak, Mohammad Shoeybi +21 · 110 citations
    Computer Science · #Computation and Language (cs.CL) #Distributed #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Parallel #Topic Modeling #and Cluster Computing (cs.DC)
  6. VILA: On Pre-training for Visual Language Models
    2023/12/12 by Lin Ji, Hongxu Yin, Lin, Ji +17 · 130 citations
    Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  7. Reducing Activation Recomputation in Large Transformer Models
    2022/05/10 by Vijay Anand Korthikanti, Jared Casper, Korthikanti, Vijay +11 · 58 citations
    Computer Science · #Advanced Neural Network Applications #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel Computing and Optimization Techniques #Topic Modeling
  8. FP8 Formats for Deep Learning
    2022/09/12 by Paulius Micikevicius, Dušan Stošić, Micikevicius, Paulius +29 · 3 voices · 31 citations
    Computer Science · Engineering · #Advanced Neural Network Applications #CCD and CMOS Imaging Sensors #Neural Networks and Applications #cs.LG
  9. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
    2022/01/28 by Shaden Smith, Smith, Shaden, Mostofa Patwary +36 · 26 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  10. RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs
    2024/07/02 by Yue Yu, Wei Ping, Yu, Yue +13 · 46 citations
    Computer Science · #Natural Language Processing Techniques #Semantic Web and Ontologies #Data Mining Algorithms and Applications
  11. Pretraining Large Language Models with NVFP4
    2025/09/29 by NVIDIA, Felix Abecassis, Anjulie Agrusa +175 · 4 voices · 13 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.AI #cs.CL #cs.LG
  12. Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset
    2024/12/03 by Dan Su, Su, Dan, Kezhi Kong +15 · 50 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Image Processing and 3D Reconstruction #Natural Language Processing Techniques #Semantic Web and Ontologies
  13. Factuality Enhanced Language Models for Open-Ended Text Generation
    2022/06/09 by Nayeon Lee, Lee, Nayeon, Ping, Wei +9 · 20 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Software Engineering Research #Topic Modeling
  14. Compact Language Models via Pruning and Knowledge Distillation
    2024/07/19 by Saurav Muralidharan, Sharath Turuvekere Sreenivas, Muralidharan, Saurav +15 · 27 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques
  15. ChatQA: Surpassing GPT-4 on Conversational QA and RAG
    2024/01/18 by Zihan Liu, Liu, Zihan, Rajarshi Roy +9 · 15 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  16. AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling
    2024/12/19 by Zihan Liu, Yang Chen, Liu, Zihan +7 · 26 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Machine Learning (cs.LG)
  17. NVLM: Open Frontier-Class Multimodal LLMs
    2024/09/17 by Wei‐Min Dai, Dai, Wenliang, Nayeon Lee +17 · 19 citations
    Arts and Humanities · Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimedia (cs.MM) #Natural Language Processing Techniques #Semantic Web and Ontologies #Translation Studies and Practices
  18. Long-Short Transformer: Efficient Transformers for Language and Vision
    2021/07/05 by Chen Zhu, Zhu, Chen, Wei Ping +11 · 8 citations
    Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Domain Adaptation and Few-Shot Learning
  19. Nemotron-4 15B Technical Report
    2024/02/26 by Jupinder Parmar, Shrimai Prabhumoye, Parmar, Jupinder +51 · 1 voice · 3 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.AI #cs.CL #cs.LG
  20. Upcycling Large Language Models into Mixture of Experts
    2024/10/10 by Ethan He, He, Ethan, Abhinav Khattar +17 · 14 citations
    Computer Science · #Expert finding and Q&A systems #Topic Modeling
  21. BioMegatron: Larger Biomedical Domain Language Model
    2020/10/12 by Hoo-Chang Shin, Shin, Hoo-Chang, Yang Zhang +11 · 5 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling
  22. Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning
    2023/02/09 by Zhuolin Yang, Yang, Zhuolin, Zihan Liu +28 · 6 citations
    Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
  23. AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy
    2025/06/16 by Zihan Liu, Zhuolin Yang, Liu, Zihan +11 · 32 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Teaching and Learning Programming
  24. Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
    2023/04/13 by Boxin Wang, Wang, Boxin, Ping, Wei +20 · 5 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
  25. Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models
    2024/07/09 by Jupinder Parmar, Parmar, Jupinder, Sanjev Satheesh +7 · 8 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques
  26. RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models
    2023/08/15 by Jie Huang, Huang, Jie, Ping, Wei +8 · 4 citations
    Computer Science · #Topic Modeling #Multimodal Machine Learning Applications #Natural Language Processing Techniques
  27. Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining
    2024/12/18 by Feng, Steven, Shrimai Prabhumoye, Kezhi Kong +10 · 9 citations
    Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Fault Detection and Control Systems #Machine Learning (cs.LG) #Machine Learning and Data Classification #Neural Networks and Applications
  28. Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
    2025/05/26 by Jaehun Jung, Jung, Jaehun, Seungju Han +17 · 10 citations
    Engineering · #Reservoir Engineering and Simulation Methods
  29. Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
    2022/02/08 by Boxin Wang, Wang, Boxin, Chaowei Xiao +14 · 2 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Adversarial Robustness in Machine Learning
  30. Scalable Training of Mixture-of-Experts Models with Megatron Core
    2026/03/08 by Zijie Yan, Hongxiao Bai, Xin Yao +42 · 3 voices · 2 citations
    #cs.DC #cs.CL #cs.LG
  31. MIND: Math Informed syNthetic Dialogues for Pretraining LLMs
    2024/10/15 by Syeda Nahida Akter, Akter, Syeda Nahida, Shrimai Prabhumoye +13 · 4 citations
    Computer Science · #Natural Language Processing Techniques #Mathematics, Computing, and Information Processing #Semantic Web and Ontologies
  32. Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning
    2025/04/06 by Ximing Lu, Lu, Ximing, Seungju Han +19 · 5 citations
    Computer Science · #Intelligent Tutoring Systems and Adaptive Learning #Explainable Artificial Intelligence (XAI) #Advanced Graph Neural Networks
  33. Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
    2026/03/19 by Zhuolin Yang, Zihan Liu, Yang Chen +14 · 3 voices
    Computer Science · #cs.CL #cs.AI #cs.LG
  34. Minitron-SSM: Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
    2025/04/15 by Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Taghibakhshi, Ali +33 · 5 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Speech Recognition and Synthesis
  35. Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data
    2025/09/26 by Syeda Nahida Akter, Shrimai Prabhumoye, Akter, Syeda Nahida +11 · 7 citations
    Computer Science · #Intelligent Tutoring Systems and Adaptive Learning
  36. Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
    2025/12/23 by NVIDIA, Aaron Blakeman, : +623 · 1 voice · 2 citations
    #cs.CL #cs.AI #cs.LG
  37. On Data Engineering for Scaling LLM Terminal Capabilities
    2026/02/24 by Renjie Pi, Grace Lam, Mohammad Shoeybi +3 · 1 voice
    Computer Science · #cs.CL
  38. Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
    2025/08/20 by Rabeeh Karimi Mahabadi, Sanjeev Satheesh, Mahabadi, Rabeeh Karimi +9 · 1 voice · 5 citations
    #cs.CL #cs.AI #cs.LG
  39. Style Example-Guided Text Generation using Generative Adversarial Transformers
    2020/03/02 by Kuo-Hao Zeng, Zeng, Kuo-Hao, Mohammad Shoeybi +3 · 1 citation
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  40. RLP: Reinforcement as a Pretraining Objective
    2025/09/26 by Ali Hatamizadeh, Hatamizadeh, Ali, Syeda Nahida Akter +13 · 4 citations
    Psychology · #Behavioral and Psychological Studies
  41. SOAP, Muon, and Beyond: Pushing LLM Pretraining Scales
    2026/07/13 by Mikail Khona, Aditya Vavre, Boxiang Wang +11
    #cs.LG #cs.AI
  42. Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos
    2026/07/17 by Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar +19
    #eess.AS #cs.CV