vix.ing · top · new · best · stats · spec

Dan Alistarh

  1. SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
    2023/01/02 by Elias Frantar, Frantar, Elias, Dan Alistarh +1 · 3 voices · 174 citations
    Computer Science · Engineering · #Advanced Data Storage Technologies #Ferroelectric and Negative Capacitance Devices #Topic Modeling #cs.LG
  2. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
    2022/10/31 by Elias Frantar, Frantar, Elias, Saleh Ashkboos +5 · 279 citations
    Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Speech Recognition and Synthesis #Topic Modeling
  3. QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models
    2023/10/25 by Elias Frantar, Dan Alistarh, Frantar, Elias +1 · 3 voices · 4 citations
    Computer Science · #Machine Learning and Algorithms #Speech Recognition and Synthesis #Topic Modeling #cs.LG
  4. SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
    2023/06/05 by Tim Dettmers, Dettmers, Tim, Ruslan Svirschevski +15 · 1 voice · 33 citations
    Computer Science · Engineering · #Ferroelectric and Negative Capacitance Devices #Speech Recognition and Synthesis #Topic Modeling #cs.CL #cs.LG
  5. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
    2024/03/30 by Saleh Ashkboos, Amirkeivan Mohtashami, Ashkboos, Saleh +14 · 107 citations
    Computer Science · Engineering · #Advanced Wireless Communication Techniques #Error Correcting Code Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Optical Network Technologies
  6. Pushing the Limits of Large Language Model Quantization via the Linearity Theorem
    2024/11/26 by Vladimir Malinovskii, Andrei Panferov, Malinovskii, Vladimir +10 · 6 voices · 5 citations
    Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.LG
  7. Sparsity in Deep Learning: Pruning and growth for efficient inference\n and training in neural networks
    2021/01/31 by Torsten Hoefler, Hoefler, Torsten, Dan Alistarh +7 · 48 citations
    Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Machine Learning (cs.LG) #Machine Learning and ELM #Neural and Evolutionary Computing (cs.NE)
  8. Model compression via distillation and quantization
    2018/02/15 by Antonio Polino, Razvan Pascanu, Polino, Antonio +3 · 29 citations
    Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE)
  9. Extreme Compression of Large Language Models via Additive Quantization
    2024/01/11 by Vage Egiazarian, Egiazarian, Vage, Andrei Panferov +9 · 29 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Speech Recognition and Synthesis
  10. The Convergence of Sparsified Gradient Methods
    2018/09/27 by Dan Alistarh, Torsten Hoefler, Alistarh, Dan +9 · 13 citations
    Computer Science · Engineering · #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel #Privacy-Preserving Technologies in Data #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
  11. Position: It's Time to Act on the Risk of Efficient Personalized Text Generation
    2025/02/10 by Eugenia Iofinova, Andrej Jovanovic, Iofinova, Eugenia +3 · 1 voice · 1 citation
    Computer Science · #Natural Language Processing Techniques #cs.CL #cs.CY
  12. Byzantine Stochastic Gradient Descent
    2018/03/23 by Dan Alistarh, Zeyuan Allen-Zhu, Alistarh, Dan +3 · 5 citations
    Computer Science · Mathematics · #Data Structures and Algorithms (cs.DS) #Distributed #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Optimization and Control (math.OC) #Parallel #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
  13. MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
    2024/08/21 by Elias Frantar, Frantar, Elias, Roberto L. Castro +7 · 11 citations
    Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Natural Language Processing Techniques #Topic Modeling
  14. Byzantine-Resilient Non-Convex Stochastic Gradient Descent
    2020/12/28 by Zeyuan Allen-Zhu, Faeze Ebrahimian, Allen-Zhu, Zeyuan +5 · 5 citations
    Computer Science · #Stochastic Gradient Optimization Techniques #Privacy-Preserving Technologies in Data #Adversarial Robustness in Machine Learning
  15. PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression
    2024/05/23 by Vladimir Malinovskii, Malinovskii, Vladimir, Denis Mazur +13 · 9 citations
    Energy · #Photovoltaic System Optimization Techniques
  16. SparCML: High-Performance Sparse Communication for Machine Learning
    2018/02/22 by Cédric Renggli, Saleh Ashkboos, Renggli, Cedric +7 · 5 citations
    Computer Science · Engineering · #Advanced Graph Neural Networks #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (stat.ML) #Parallel #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
  17. LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics
    2024/10/21 by Thomas Robert, Mher Safaryan, Robert, Thomas +5 · 8 citations
    Computer Science · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
  18. Adaptive Gradient Quantization for Data-Parallel SGD
    2020/10/23 by Fartash Faghri, Faghri, Fartash, Iman Tabrizian +9 · 3 citations
    Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Medical Image Segmentation Techniques
  19. How to Elect a Leader Faster than a Tournament
    2014/11/04 by Dan Alistarh, Rati Gelashvili, Alistarh, Dan +3 · 1 voice
    Computer Science · #Distributed systems and fault tolerance #Optimization and Search Problems #Cryptography and Data Security
  20. SSSE: Efficiently Erasing Samples from Trained Machine Learning Models
    2021/07/08 by Alexandra Peşte, Peste, Alexandra, Dan Alistarh +3 · 3 citations
    Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification
  21. SPDY: Accurate Pruning with Speedup Guarantees
    2022/01/31 by Elias Frantar, Dan Alistarh, Frantar, Elias +1 · 3 citations
    Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
  22. Space-Optimal Majority in Population Protocols
    2017/04/17 by Dan Alistarh, James Aspnes, Alistarh, Dan +3 · 2 citations
    Computer Science · Engineering · #Distributed systems and fault tolerance #Privacy-Preserving Technologies in Data #Modular Robots and Swarm Intelligence
  23. Optimizers Qualitatively Alter Solutions And We Should Leverage This
    2025/07/16 by Razvan Pascanu, Pascanu, Razvan, Clare Lyle +17 · 2 voices · 7 citations
    Computer Science · #Stochastic Gradient Optimization Techniques #Advanced Neural Network Applications #Explainable Artificial Intelligence (XAI)
  24. Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
    2024/04/04 by Aniruddha Nrusimha, Nrusimha, Aniruddha, Mayank Mishra +9 · 4 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
  25. Scaling Laws for Sparsely-Connected Foundation Models
    2023/09/15 by Elias Frantar, Frantar, Elias, Carlos Riquelme +7 · 3 citations
    Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
  26. CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models
    2022/10/14 by Denis Kuznedelev, Eldar Kurtic, Kuznedelev, Denis +5 · 2 citations
    Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Advanced Image and Video Retrieval Techniques
  27. Near-Optimal Leader Election in Population Protocols on Graphs
    2022/05/25 by Dan Alistarh, Joel Rybicki, Alistarh, Dan +3 · 2 citations
    Computer Science · #Cooperative Communication and Network Coding #Distributed systems and fault tolerance #Caching and Content Delivery
  28. Knowledge Distillation Performs Partial Variance Reduction
    2023/05/27 by Mher Safaryan, Safaryan, Mher, Alexandra Peşte +3 · 2 citations
    Computer Science · Physics and Astronomy · #Stochastic Gradient Optimization Techniques #Model Reduction and Neural Networks #Domain Adaptation and Few-Shot Learning
  29. "Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization
    2024/11/04 by Eldar Kurtic, Kurtic, Eldar, Alexandre Carriconde Marques +7 · 4 citations
    Engineering · Computer Science · #Reservoir Engineering and Simulation Methods #Distributed and Parallel Computing Systems
  30. In Search of the Fastest Concurrent Union-Find Algorithm
    2019/11/14 by Dan Alistarh, Alistarh, Dan, Alexander Fedorov +3 · 1 voice
    #cs.DS
  31. WoodFisher: Efficient Second-Order Approximation for Neural Network Compression
    2020/04/29 by Sidak Pal Singh, Dan Alistarh, Singh, Sidak Pal +1 · 1 citation
    Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Multimodal Machine Learning Applications
  32. Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment
    2024/05/06 by Abhinav Agarwalla, Abhay Gupta, Agarwalla, Abhinav +21 · 2 citations
    Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computational Geometry and Mesh Generation #Dynamics and Control of Mechanical Systems #FOS: Computer and information sciences #Robotic Mechanisms and Dynamics
  33. M-FAC: Efficient Matrix-Free Approximations of Second-Order Information
    2021/07/07 by Elias Frantar, Eldar Kurtic, Frantar, Elias +3 · 1 citation
    Computer Science · Mathematics · Engineering · #Stochastic Gradient Optimization Techniques #Tensor decomposition and applications #Sparse and Compressive Sensing Techniques
  34. EvoPress: Accurate Dynamic Model Compression via Evolutionary Search
    2024/10/18 by Oliver Sieberling, Sieberling, Oliver, Denis Kuznedelev +5 · 2 citations
    Computer Science · #Evolutionary Algorithms and Applications #Reinforcement Learning in Robotics
  35. Multi-Queues Can Be State-of-the-Art Priority Schedulers
    2021/09/02 by Anastasiia Postnikova, Nikita Koval, Postnikova, Anastasiia +5 · 1 citation
    Computer Science · #Cloud Computing and Resource Management #Data Structures and Algorithms (cs.DS) #Distributed #Distributed systems and fault tolerance #FOS: Computer and information sciences #Parallel #Parallel Computing and Optimization Techniques #and Cluster Computing (cs.DC)
  36. The Iterative Optimal Brain Surgeon: Faster Sparse Recovery by Leveraging Second-Order Information
    2024/08/30 by D.-S. Wu, Ionuț-Vlad Modoranu, Wu, Diyuan +7 · 2 citations
    Medicine · Physics and Astronomy · #Advanced MRI Techniques and Applications #Atomic and Subatomic Physics Research #FOS: Computer and information sciences #Machine Learning (cs.LG)
  37. GMP*: Well-Tuned Gradual Magnitude Pruning Can Outperform Most BERT-Pruning Methods
    2022/10/12 by Eldar Kurtic, Dan Alistarh, Kurtic, Eldar +1 · 1 citation
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
  38. Vision Models Can Be Efficiently Specialized via Few-Shot Task-Aware Compression
    2023/03/25 by Denis Kuznedelev, Soroush Tabesh, Kuznedelev, Denis +11 · 1 citation
    Computer Science · #68T07 #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #I.m
  39. DarwinLM: Evolutionary Structured Pruning of Large Language Models
    2025/02/11 by Shengkun Tang, Oliver Sieberling, Tang, Shengkun +7 · 3 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling
  40. Efficient Self-Adjusting Search Trees via Lazy Updates
    2023/10/08 by Alexander Slastin, Dan Alistarh, Slastin, Alexander +3 · 1 citation
    Computer Science · #Algorithms and Data Compression #Distributed systems and fault tolerance #Cloud Computing and Resource Management
  41. Towards Combinatorial Interpretability of Neural Computation
    2025/04/10 by Micah Adler, Dan Alistarh, Adler, Micah +3 · 3 citations
    Computer Science · #Neural Networks and Applications #Statistical and Computational Modeling #Explainable Artificial Intelligence (XAI)
  42. Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models
    2024/06/18 by Eldar Kurtic, Kurtic, Eldar, Amir Moeini +3 · 1 citation
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #I.2.7 #Intelligent Tutoring Systems and Adaptive Learning #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling
  43. The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
    2025/07/24 by Jiale Chen, Chen, Jiale, Yalda Shabanzadeh +7 · 3 citations
    Computer Science · Mathematics · #Advanced Data Compression Techniques #Advanced Vision and Imaging #Image and Object Detection Techniques #cs.DS #cs.IT #cs.LG #math.IT
  44. Communication-Efficient Distributed Optimization with Quantized Preconditioners
    2021/02/14 by Foivos Alimisis, Peter Maxwell Davies, Alimisis, Foivos +3 · 1 citation
    Computer Science · Engineering · #Complexity and Algorithms in Graphs #Distributed #FOS: Computer and information sciences #FOS: Mathematics #Optimization and Control (math.OC) #Parallel #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
  45. GPTQ-2D: Cubic-Time Two-Sided Adaptive Rounding
    2026/07/29 by Jiale Chen, Torsten Hoefler, Dan Alistarh
    Computer Science · #cs.DS #cs.LG