Ge, Keshi
- Oases: Efficient Large-Scale Model Training on Commodity Servers via Overlapped and Automated Tensor Model Parallelism
2023/05/25 by Li, Shengwei, Lai, Zhiquan, Li, Dongsheng +5 · 2 citations
#Distributed #FOS: Computer and information sciences #Parallel #and Cluster Computing (cs.DC)
- Pro-Prophet: A Systematic Load Balancing Method for Efficient Parallel Training of Large-scale MoE Models
2024/11/15 by Wei Wang, Zhiquan Lai, Wang, Wei +11 · 2 citations
Computer Science · #Distributed and Parallel Computing Systems #Software System Performance and Reliability #Cloud Computing and Resource Management