Dettmers, Tim
- QLoRA: Efficient Finetuning of Quantized LLMs
2023/05/23 by Tim Dettmers, Artidoro Pagnoni, Dettmers, Tim +5 · 3 voices · 518 citations
#cs.LG
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
2022/08/15 by Tim Dettmers, Mike Lewis, Dettmers, Tim +5 · 5 voices · 96 citations
Computer Science · Engineering · #Ferroelectric and Negative Capacitance Devices #Parallel Computing and Optimization Techniques #Topic Modeling #cs.AI #cs.LG
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
2023/06/05 by Tim Dettmers, Dettmers, Tim, Ruslan Svirschevski +15 · 1 voice · 35 citations
Computer Science · Engineering · #Ferroelectric and Negative Capacitance Devices #Speech Recognition and Synthesis #Topic Modeling #cs.CL #cs.LG
- BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
2022/11/09 by Workshop, BigScience, :, Scao, Teven Le +391 · 109 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences
- MatFormer: Nested Transformer for Elastic Inference
2023/10/11 by Devvrit, Sneha Kudugunta, Aditya Kusupati +20 · 5 voices · 20 citations
Computer Science · Social Sciences · #Caching and Content Delivery #Human Mobility and Location-Based Analysis #Opportunistic and Delay-Tolerant Networks #cs.CL #cs.CV #cs.LG
- Petals: Collaborative Inference and Fine-tuning of Large Models
2022/09/02 by Alexander Borzunov, Borzunov, Alexander, Dmitry Baranchuk +13 · 2 voices · 7 citations
Computer Science · #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC) #cs.DC #cs.LG
- Convolutional 2D Knowledge Graph Embeddings
2017/07/05 by Tim Dettmers, Pasquale Minervini, Dettmers, Tim +5 · 37 citations
Computer Science · Decision Sciences · #Advanced Graph Neural Networks #Data Quality and Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling
- 8-bit Optimizers via Block-wise Quantization
2021/10/06 by Tim Dettmers, Dettmers, Tim, Mike Lewis +5 · 48 citations
Computer Science · Mathematics · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Stochastic Gradient Optimization Techniques #Tensor decomposition and applications
- OLMoE: Open Mixture-of-Experts Language Models
2024/09/03 by Niklas Muennighoff, Muennighoff, Niklas, Luca Soldaini +45 · 1 voice · 61 citations
Computer Science · #Expert finding and Q&A systems #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG
- BASE Layers: Simplifying Training of Large, Sparse Models
2021/03/30 by Mike Lewis, Shruti Bhosale, Lewis, Mike +7 · 33 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Machine Learning and Algorithms
- The case for 4-bit precision: k-bit Inference Scaling Laws
2022/12/19 by Tim Dettmers, Dettmers, Tim, Luke Zettlemoyer +1 · 24 citations
Computer Science · Engineering · #Topic Modeling #Ferroelectric and Negative Capacitance Devices #Machine Learning and Algorithms
- 8-Bit Approximations for Parallelism in Deep Learning
2015/11/14 by Tim Dettmers, Dettmers, Tim · 15 citations
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neural Networks and Reservoir Computing #Neural and Evolutionary Computing (cs.NE) #Sparse and Compressive Sensing Techniques
- Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models
2022/08/05 by Margaret Li, Li, Margaret, Suchin Gururangan +11 · 2 voices · 10 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL
- Stable and low-precision training for large-scale vision-language models
2023/04/25 by Mitchell Wortsman, Tim Dettmers, Wortsman, Mitchell +9 · 15 citations
Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Natural Language Processing Techniques
- Sparse Networks from Scratch: Faster Training without Losing Performance
2019/07/10 by Dettmers, Tim, Zettlemoyer, Luke · 8 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)
- Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
2024/07/09 by Shao, Rulin, He, Jacqueline, Asai, Akari +5 · 18 citations
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG)
- Distributed Inference and Fine-tuning of Large Language Models Over The Internet
2023/12/13 by Alexander Borzunov, Borzunov, Alexander, Max Ryabinin +13 · 11 citations
Computer Science · Engineering · #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Parallel #Privacy-Preserving Technologies in Data #Topic Modeling #and Cluster Computing (cs.DC)
- SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
2023/01/27 by Max Ryabinin, Tim Dettmers, Ryabinin, Max +5 · 6 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Parallel #Topic Modeling #and Cluster Computing (cs.DC)
- Holistically Evaluating the Environmental Impact of Creating Language Models
2025/03/03 by Jacob Morrison, Clara Na, Morrison, Jacob +9 · 3 voices · 10 citations
#cs.CY #cs.AI #cs.LG
- Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model
2023/05/23 by Liu, Zeyu Leo, Dettmers, Tim, Lin, Xi Victoria +2 · 1 citation
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)