Demmel, James
- Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
2019/04/01 by Yang You, You, Yang, Jing Li +17 · 41 citations
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Stochastic Gradient Optimization Techniques
- Randomized Numerical Linear Algebra : A Perspective on the Field With an Eye to Software
2023/02/22 by Riley Murray, James Demmel, Murray, Riley +23 · 9 citations
Computer Science · Mathematics · #Stochastic Gradient Optimization Techniques #Tensor decomposition and applications #Parallel Computing and Optimization Techniques
- An Efficient Sparse Kernel Generator for O(3)-Equivariant Deep Networks
2025/01/23 by Vivek Bharadwaj, Austin Glover, Bharadwaj, Vivek +6 · 1 voice · 8 citations
Computer Science · Earth and Planetary Sciences · Engineering · #Medical Image Segmentation Techniques #Medical Imaging and Analysis #Seismic Imaging and Inversion Techniques #cs.AI #cs.LG
- CoSA: Scheduling by Constrained Optimization for Spatial Accelerators
2021/05/05 by Huang, Qijing, Kang, Minwoo, Dinh, Grace +5 · 4 citations
#FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Machine Learning (cs.LG)
- Communication lower bounds and optimal algorithms for programs that reference arrays -- Part 1
2013/08/01 by Michael Christ, Christ, Michael, James Demmel +7 · 2 citations
Engineering · #26D15 #68W40 #Classical Analysis and ODEs (math.CA) #Computational Complexity (cs.CC) #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Sparse and Compressive Sensing Techniques #Topology Optimization in Engineering #VLSI and FPGA Design Techniques
- Communication-optimal parallel and sequential QR and LU factorizations
2008/08/19 by James Demmel, Laura Grigori, Demmel, James +5 · 2 citations
Computer Science · #65F05 #FOS: Mathematics #Interconnection Networks and Systems #Numerical Analysis (math.NA) #Parallel Computing and Optimization Techniques #Stochastic Gradient Optimization Techniques
- Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1\n Accuracy in One Hour
2020/10/30 by Arissa Wongpanich, Hieu Pham, Wongpanich, Arissa +12 · 1 voice
Computer Science · Medicine · Neuroscience · #Advanced Neural Network Applications #Brain Tumor Detection and Classification #COVID-19 diagnosis using AI #cs.CV #cs.DC #cs.LG
- Parallelepipeds obtaining HBL lower bounds
2016/11/18 by Demmel, James, Rusciano, Alex · 1 citation
#Classical Analysis and ODEs (math.CA) #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics
- ImageNet Training in Minutes
2017/09/14 by You, Yang, Zhang, Zhao, Hsieh, Cho-Jui +2 · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences
- Large Scale Multi-GPU Based Parallel Traffic Simulation for Accelerated Traffic Assignment and Propagation
2024/04/25 by Xuan Jiang, Jiang, Xuan, Raja Sengupta +5 · 3 citations
Decision Sciences · Engineering · #Distributed #FOS: Computer and information sciences #Parallel #Simulation Techniques and Applications #Traffic Prediction and Management Techniques #and Cluster Computing (cs.DC)
- The Limit of the Batch Size
2020/06/15 by You, Yang, Wang, Yuhui, Zhang, Huan +3 · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel #and Cluster Computing (cs.DC)
- Avoiding Synchronization in First-Order Methods for Sparse Convex Optimization
2017/12/17 by Devarakonda, Aditya, Fountoulakis, Kimon, Demmel, James +1 · 1 citation
#68W10 #90C25 #Distributed #FOS: Computer and information sciences #FOS: Mathematics #G.1.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Parallel #and Cluster Computing (cs.DC)
- Distributed-Memory Sparse Kernels for Machine Learning
2022/03/15 by Bharadwaj, Vivek, Buluç, Aydin, Demmel, James · 1 citation
#Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)
- Fast Exact Leverage Score Sampling from Khatri-Rao Products with Applications to Tensor Decomposition
2023/01/29 by Bharadwaj, Vivek, Malik, Osman Asif, Murray, Riley +3 · 1 citation
#FOS: Mathematics #Numerical Analysis (math.NA)
- CholeskyQR with Randomization and Pivoting for Tall Matrices (CQRRPT)
2023/11/14 by Maksim Melnichenko, Melnichenko, Maksim, Oleg Balabanov +9 · 1 citation
Computer Science · Physics and Astronomy · Mathematics · #Matrix Theory and Algorithms #Advanced Mathematical Theories and Applications #Graph theory and applications
- Fast and Inverse-Free Algorithms for Deflating Subspaces
2023/09/29 by Demmel, James, Dumitriu, Ioana, Schneider, Ryan · 1 citation
#65F60 #FOS: Mathematics #Numerical Analysis (math.NA)
- StarTrail: Concentric Ring Sequence Parallelism for Efficient Near-Infinite-Context Transformer Model Training
2024/06/30 by Liu Ziming, Liu, Ziming, Shaoyu Wang +12 · 1 citation
Computer Science · Physics and Astronomy · #Computational Physics and Python Applications #Distributed #FOS: Computer and information sciences #Gamma-ray bursts and supernovae #Magnetic confinement fusion research #Parallel #and Cluster Computing (cs.DC)
- Avoiding Communication in Logistic Regression
2020/11/16 by Devarakonda, Aditya, Demmel, James · 1 citation
#Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)
- Minimizing the Arithmetic and Communication Complexity of Jacobi's Method for Eigenvalues and Singular Values: Part One -- Serial Algorithms
2025/06/04 by James Demmel, Demmel, James, Hengrui Luo +5 · 1 citation
Computer Science · #Stochastic Gradient Optimization Techniques #Polynomial and algebraic computation #Matrix Theory and Algorithms