Tanaka, Masahiro
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
2024/04/22 by Marah Abdin, Jyoti Aneja, Abdin, Marah +262 · 8 voices · 364 citations
Computer Science · #Computational Physics and Python Applications
- DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
2023/09/25 by Sam Adé Jacobs, Jacobs, Sam Ade, Masahiro Tanaka +11 · 48 citations
Computer Science · Engineering · #Topic Modeling #Ferroelectric and Negative Capacitance Devices #Natural Language Processing Techniques
- DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
2023/08/02 by Zhewei Yao, Reza Yazdani Aminabadi, Yao, Zhewei +35 · 16 citations
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification
- DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
2024/01/09 by Connor Holmes, Masahiro Tanaka, Holmes, Connor +19 · 19 citations
Computer Science · Engineering · #Topic Modeling #Natural Language Processing Techniques #Ferroelectric and Negative Capacitance Devices
- DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
2023/10/06 by Shuaiwen Leon Song, Bonnie Kruft, Song, Shuaiwen Leon +181 · 4 citations
Biochemistry, Genetics and Molecular Biology · Decision Sciences · Materials Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Genetics, Bioinformatics, and Biomedical Research #Machine Learning (cs.LG) #Machine Learning in Materials Science #Scientific Computing and Data Management
- An Analysis of Concurrency Control Protocols for In-Memory Databases with CCBench (Extended Version)
2020/09/24 by Tanabe, Takayuki, Hoshino, Takashi, Kawashima, Hideyuki +3 · 2 citations
#Databases (cs.DB) #FOS: Computer and information sciences #H.2.4
- Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelis
2024/06/27 by Lian, Xinyu, Jacobs, Sam Ade, Kurilenko, Lev +4 · 3 citations
#Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)
- Automatic Graph Partitioning for Very Large-scale Deep Learning
2021/03/30 by Masahiro Tanaka, Tanaka, Masahiro, Kenjiro Taura +5 · 1 citation
Computer Science · Engineering · #Advanced Graph Neural Networks #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)
- Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer
2024/08/30 by Jinghan Yao, Yao, Jinghan, Sam Adé Jacobs +9 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Parallel #Topic Modeling #and Cluster Computing (cs.DC)
- LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
2024/07/04 by LLM-Jp, :, Aizawa, Akiko +80 · 2 citations
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences
- ZenFlow: Enabling Stall-Free Offloading Training via Asynchronous Updates
2025/05/18 by Lan, Tingfeng, Wu, Yusen, Ma, Bin +7 · 3 citations
#C.1.4 #D.4.7 #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)
- GRIN: GRadient-INformed MoE
2024/09/18 by Liyuan Liu, Liu, Liyuan, Young Jin Kim +30 · 1 citation
Computer Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling