vix.ing · top · new · best · stats · spec

Shi, Shaohuai

  1. Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
    2018/07/30 by Xianyan Jia, Shutao Song, Jia, Xianyan +26 · 1 voice · 42 citations
    Computer Science · Medicine · #cs.LG #cs.DC #stat.ML
  2. Benchmarking State-of-the-Art Deep Learning Software Tools
    2016/08/25 by Shaohuai Shi, Qiang Wang, Shi, Shaohuai +5 · 1 voice · 8 citations
    Computer Science · Engineering · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Ferroelectric and Negative Capacitance Devices
  3. LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuning
    2023/08/07 by Longteng Zhang, Zhang, Longteng, Lin Zhang +7 · 33 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
  4. Performance Modeling and Evaluation of Distributed Deep Learning Frameworks on GPUs
    2017/11/16 by Shaohuai Shi, Qiang Wang, Shi, Shaohuai +3 · 1 voice · 1 citation
    Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Stochastic Gradient Optimization Techniques #cs.DC
  5. Understanding Top-k Sparsification in Distributed Deep Learning
    2019/11/20 by Shaohuai Shi, Xiaowen Chu, Shi, Shaohuai +5 · 6 citations
    Computer Science · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Parallel #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
  6. FADNet: A Fast and Accurate Network for Disparity Estimation
    2020/03/24 by Qiang Wang, Wang, Qiang, Shaohuai Shi +7 · 5 citations
    Computer Science · #Advanced Image Processing Techniques #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Enhancement Techniques #Machine Learning (cs.LG)
  7. ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
    2024/10/23 by Xin He, Shunkang Zhang, He, Xin +14 · 11 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Expert finding and Q&A systems #FOS: Computer and information sciences #Mobile Crowdsensing and Crowdsourcing #Seismology and Earthquake Studies
  8. Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
    2020/03/10 by Zhenheng Tang, Tang, Zhenheng, Shaohuai Shi +7 · 4 citations
    Computer Science · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Parallel #Privacy-Preserving Technologies in Data #Signal Processing (eess.SP) #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC) #electronic engineering #information engineering
  9. A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networks
    2019/01/14 by Shi, Shaohuai, Wang, Qiang, Zhao, Kaiyong +4 · 2 citations
    #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)
  10. Evaluation and Optimization of Gradient Compression for Distributed Deep Learning
    2023/06/15 by Lin Zhang, Longteng Zhang, Zhang, Lin +7 · 3 citations
    Computer Science · Engineering · #Distributed #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Parallel #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
  11. Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters
    2020/10/20 by Shi, Shaohuai, Zhou, Xianhao, Song, Shutao +21 · 2 citations
    #Artificial Intelligence (cs.AI) #Distributed #FOS: Computer and information sciences #Parallel #and Cluster Computing (cs.DC)
  12. Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing
    2024/07/18 by Luo, Yizhou, Wang, Qiang, Shi, Shaohuai +4 · 3 citations
    #Distributed #FOS: Computer and information sciences #Parallel #and Cluster Computing (cs.DC)
  13. Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
    2022/06/06 by Zhenheng Tang, Yonggang Zhang, Tang, Zhenheng +9 · 2 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Data Stream Mining Techniques #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
  14. MG-WFBP: Efficient Data Communication for Distributed Synchronous SGD Algorithms
    2018/11/27 by Shi, Shaohuai, Chu, Xiaowen, Li, Bo · 1 citation
    #Distributed #FOS: Computer and information sciences #Parallel #and Cluster Computing (cs.DC)
  15. Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
    2019/11/20 by Shaohuai Shi, Zhenheng Tang, Shi, Shaohuai +7 · 1 citation
    Computer Science · Engineering · #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Parallel #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
  16. Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
    2020/02/22 by Tang, Zhenheng, Shi, Shaohuai, Chu, Xiaowen · 1 citation
    #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel #and Cluster Computing (cs.DC)
  17. Supervised Learning Based Algorithm Selection for Deep Neural Networks
    2017/02/10 by Shaohuai Shi, Pengfei Xu, Shi, Shaohuai +3 · 1 voice
    Computer Science · #Advanced Neural Network Applications #Machine Learning and Data Classification #Machine Learning and ELM #cs.DC #cs.LG
  18. FedImpro: Measuring and Improving Client Update in Federated Learning
    2024/02/10 by Zhenheng Tang, Tang, Zhenheng, Yonggang Zhang +11 · 2 citations
    Computer Science · Social Sciences · #Access Control and Trust #Artificial Intelligence (cs.AI) #Cloud Data Security Solutions #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #Privacy-Preserving Technologies in Data #and Cluster Computing (cs.DC)
  19. Dissecting the Runtime Performance of the Training, Fine-tuning, and Inference of Large Language Models
    2023/11/07 by Longteng Zhang, Zhang, Longteng, Xiang Liu +19 · 2 citations
    Computer Science · Engineering · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Natural Language Processing Techniques #Performance (cs.PF) #Topic Modeling
  20. FADNet++: Real-Time and Accurate Disparity Estimation with Configurable Networks
    2021/10/06 by Qiang Wang, Wang, Qiang, Shaohuai Shi +7 · 1 citation
    Computer Science · #Advanced Neural Network Applications #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG)
  21. DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronization
    2025/02/16 by Zhenheng Tang, Tang, Zhenheng, Zichen Tang +17 · 3 citations
    Computer Science · Engineering · Neuroscience · #Brain Tumor Detection and Classification #Cloud Computing and Resource Management #Distributed #FOS: Computer and information sciences #Parallel #Robotics and Automated Systems #and Cluster Computing (cs.DC)
  22. EASNet: Searching Elastic and Accurate Network Architecture for Stereo Matching
    2022/07/20 by Qiang Wang, Wang, Qiang, Shaohuai Shi +5 · 1 citation
    Computer Science · #Advanced Vision and Imaging #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications
  23. DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining
    2023/02/24 by Zhang Li, Shaohuai Shi, Zhang, Lin +9 · 1 citation
    Computer Science · Engineering · Neuroscience · #Advanced Neural Network Applications #Brain Tumor Detection and Classification #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)
  24. Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules
    2024/06/30 by Pan, Xinglin, Shaohuai Shi, Lin, Wenxiang +7 · 2 citations
    Computer Science · #Machine Learning and Algorithms #Advanced Neural Network Applications #Parallel Computing and Optimization Techniques
  25. Task Scheduling for Efficient Inference of Large Language Models on Single Moderate GPU Systems
    2024/11/24 by Lin, Wenxiang, Pan, Xinglin, Shi, Shaohuai +2 · 2 citations
    #Computational Engineering #FOS: Computer and information sciences #Finance #and Science (cs.CE)
  26. FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression
    2024/10/16 by Zhenheng Tang, Tang, Zhenheng, Kang, Xueze +23 · 1 citation
    Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mobile Agent-Based Network Management #Natural Language Processing Techniques #Parallel #Robotics and Automated Systems #and Cluster Computing (cs.DC)