Tim Dettmers
- QLoRA: Efficient Finetuning of Quantized LLMs
2023/05/23 by Tim Dettmers, Dettmers, Tim, Artidoro Pagnoni +5 · 3 voices · 554 citations
#cs.LG
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
2022/08/15 by Tim Dettmers, Mike Lewis, Dettmers, Tim +5 · 5 voices · 101 citations
Computer Science · Engineering · #Ferroelectric and Negative Capacitance Devices #Parallel Computing and Optimization Techniques #Topic Modeling #cs.AI #cs.LG
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
2023/06/05 by Tim Dettmers, Dettmers, Tim, Ruslan Svirschevski +15 · 1 voice · 39 citations
Computer Science · Engineering · #Ferroelectric and Negative Capacitance Devices #Speech Recognition and Synthesis #Topic Modeling #cs.CL #cs.LG
- MatFormer: Nested Transformer for Elastic Inference
2023/10/11 by Sneha Kudugunta, Devvrit, Kudugunta, Sneha +20 · 5 voices · 22 citations
Computer Science · Social Sciences · #Caching and Content Delivery #Human Mobility and Location-Based Analysis #Opportunistic and Delay-Tolerant Networks #cs.CL #cs.CV #cs.LG
- Petals: Collaborative Inference and Fine-tuning of Large Models
2022/09/02 by Alexander Borzunov, Borzunov, Alexander, Dmitry Baranchuk +13 · 2 voices · 11 citations
Computer Science · #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC) #cs.DC #cs.LG
- Convolutional 2D Knowledge Graph Embeddings
2017/07/05 by Tim Dettmers, Dettmers, Tim, Pasquale Minervini +5 · 41 citations
Computer Science · Decision Sciences · #Advanced Graph Neural Networks #Data Quality and Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling
- 8-bit Optimizers via Block-wise Quantization
2021/10/06 by Tim Dettmers, Mike Lewis, Dettmers, Tim +5 · 52 citations
Computer Science · Mathematics · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Stochastic Gradient Optimization Techniques #Tensor decomposition and applications
- OLMoE: Open Mixture-of-Experts Language Models
2024/09/03 by Niklas Muennighoff, Luca Soldaini, Muennighoff, Niklas +45 · 1 voice · 64 citations
Computer Science · #Expert finding and Q&A systems #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG
- BASE Layers: Simplifying Training of Large, Sparse Models
2021/03/30 by Mike Lewis, Shruti Bhosale, Lewis, Mike +7 · 39 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Machine Learning and Algorithms
- SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
2023/01/27 by Max Ryabinin, Tim Dettmers, Ryabinin, Max +6 · 1 voice · 10 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Parallel #Topic Modeling #and Cluster Computing (cs.DC) #cs.DC #cs.LG
- The case for 4-bit precision: k-bit Inference Scaling Laws
2022/12/19 by Tim Dettmers, Luke Zettlemoyer, Dettmers, Tim +1 · 26 citations
Computer Science · Engineering · #Topic Modeling #Ferroelectric and Negative Capacitance Devices #Machine Learning and Algorithms
- 8-Bit Approximations for Parallelism in Deep Learning
2015/11/14 by Tim Dettmers, Dettmers, Tim · 15 citations
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neural Networks and Reservoir Computing #Neural and Evolutionary Computing (cs.NE) #Sparse and Compressive Sensing Techniques
- Stable and low-precision training for large-scale vision-language models
2023/04/25 by Mitchell Wortsman, Tim Dettmers, Wortsman, Mitchell +9 · 18 citations
Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Natural Language Processing Techniques
- Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models
2022/08/05 by Margaret Li, Li, Margaret, Suchin Gururangan +11 · 2 voices · 11 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL
- Sparse Networks from Scratch: Faster Training without Losing Performance
2019/07/10 by Tim Dettmers, Dettmers, Tim, Luke Zettlemoyer +1 · 9 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)
- Distributed Inference and Fine-tuning of Large Language Models Over The Internet
2023/12/13 by Alexander Borzunov, Borzunov, Alexander, Max Ryabinin +13 · 15 citations
Computer Science · Engineering · #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Parallel #Privacy-Preserving Technologies in Data #Topic Modeling #and Cluster Computing (cs.DC)
- Holistically Evaluating the Environmental Impact of Creating Language Models
2025/03/03 by Jacob Morrison, Morrison, Jacob, Clara Na +9 · 3 voices · 11 citations
#cs.CY #cs.AI #cs.LG
- Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model
2023/05/23 by Liu, Zeyu Leo, Tim Dettmers, Dettmers, Tim +6 · 3 citations
Computer Science · Engineering · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Speech Recognition and Synthesis #Topic Modeling