Yair Carmon
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
2022/03/10 by Mitchell Wortsman, Gabriel Ilharco, Wortsman, Mitchell +19 · 3 voices · 169 citations
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification #cs.CL #cs.CV #cs.LG
- DataComp-LM: In search of the next generation of training sets for language models
2024/06/17 by Jeffrey Li, Alex Chengyu Fang, Li, Jeffrey +121 · 2 voices · 82 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.LG
- DataComp: In search of the next generation of multimodal datasets
2023/04/27 by Samir Yitzhak Gadre, Gadre, Samir Yitzhak, Gabriel Ilharco +65 · 81 citations
Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning in Healthcare #Topic Modeling
- Lower Bounds for Non-Convex Stochastic Optimization
2019/12/05 by Yossi Arjevani, Yair Carmon, Arjevani, Yossi +9 · 27 citations
Computer Science · Mathematics · Engineering · #Stochastic Gradient Optimization Techniques #Markov Chains and Monte Carlo Methods #Sparse and Compressive Sensing Techniques
- Lower bounds for finding stationary points I
2019/06/07 by Yair Carmon, John C. Duchi, Oliver Hinder +1 · 25 citations
Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Complexity and Algorithms in Graphs
- No bad local minima: Data independent training error guarantees for\n multilayer neural networks
2016/05/26 by Daniel Soudry, Yair Carmon, Soudry, Daniel +1 · 24 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE) #Stochastic Gradient Optimization Techniques
- Unlabeled Data Improves Adversarial Robustness
2019/05/31 by Yair Carmon, Aditi Raghunathan, Carmon, Yair +7 · 19 citations
Computer Science · #Adversarial Robustness in Machine Learning #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning
- Accuracy on the Line: On the Strong Correlation Between\n Out-of-Distribution and In-Distribution Generalization
2021/07/09 by J. J. Miller, Miller, John, Rohan Taori +15 · 15 citations
Computer Science · Medicine · #Advanced Neural Network Applications #COVID-19 diagnosis using AI #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification
- "Convex Until Proven Guilty": Dimension-Free Acceleration of Gradient\n Descent on Non-Convex Functions
2017/05/08 by Yair Carmon, Oliver Hinder, Carmon, Yair +5 · 15 citations
Medicine · Engineering · #Bone and Joint Diseases #Sparse and Compressive Sensing Techniques #Glioma Diagnosis and Treatment
- DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule
2023/02/08 by Maor Ivgi, Ivgi, Maor, Oliver Hinder +3 · 17 citations
Computer Science · #Stochastic Gradient Optimization Techniques #Domain Adaptation and Few-Shot Learning #Advanced Neural Network Applications
- Large-Scale Methods for Distributionally Robust Optimization
2020/10/12 by Daniel Lévy, Levy, Daniel, Yair Carmon +5 · 9 citations
Business, Management and Accounting · Decision Sciences · Engineering · #Advanced Control Systems Optimization #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Risk and Portfolio Optimization #Supply Chain and Inventory Management
- Resolving Discrepancies in Compute-Optimal Scaling of Language Models
2024/06/27 by Tomer Porian, Porian, Tomer, Mitchell Wortsman +7 · 16 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Speech Recognition and Synthesis
- Optimal and Adaptive Monteiro-Svaiter Acceleration
2022/05/30 by Yair Carmon, Danielle Hausler, Carmon, Yair +7 · 8 citations
Mathematics · Engineering · Computer Science · #Advanced Optimization Algorithms Research #Sparse and Compressive Sensing Techniques #Matrix Theory and Algorithms
- Language models scale reliably with over-training and on downstream tasks
2024/03/13 by Samir Yitzhak Gadre, Georgios Smyrnis, Gadre, Samir Yitzhak +45 · 12 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling
- Making SGD Parameter-Free
2022/05/04 by Yair Carmon, Oliver Hinder, Carmon, Yair +1 · 7 citations
Computer Science · Decision Sciences · Engineering · #Stochastic Gradient Optimization Techniques #Advanced Bandit Algorithms Research #Sparse and Compressive Sensing Techniques
- Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments
2022/02/13 by Maor Ivgi, Ivgi, Maor, Yair Carmon +3 · 4 citations
Computer Science · Materials Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning in Materials Science #Natural Language Processing Techniques #Numerical Analysis (math.NA) #Topic Modeling
- ReSQueing Parallel and Private Stochastic Convex Optimization
2023/01/01 by Yair Carmon, Carmon, Yair, Arun Jambulapati +11 · 4 citations
Computer Science · Engineering · Mathematics · #Cryptography and Security (cs.CR) #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Accelerated Parameter-Free Stochastic Optimization
2024/03/31 by Itai Kreisler, Kreisler, Itai, Maor Ivgi +5 · 5 citations
Computer Science · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Neural Networks and Applications #Optimization and Control (math.OC)
- Stochastic Bias-Reduced Gradient Methods
2021/06/17 by Hilal Asi, Asi, Hilal, Yair Carmon +7 · 3 citations
Computer Science · Engineering · #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Privacy-Preserving Technologies in Data #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Thinking Inside the Ball: Near-Optimal Minimization of the Maximal Loss
2021/05/04 by Yair Carmon, Carmon, Yair, Arun Jambulapati +5 · 2 citations
Computer Science · Decision Sciences · #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Probability and Risk Models #Risk and Portfolio Optimization #Stochastic Gradient Optimization Techniques
- Accelerated Methods for Non-Convex Optimization
2016/11/02 by Yair Carmon, John C. Duchi, Carmon, Yair +5 · 2 citations
Computer Science · Engineering · Mathematics · #Advanced Optimization Algorithms Research #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Coordinate Methods for Matrix Games
2020/09/17 by Yair Carmon, Yujia Jin, Carmon, Yair +5 · 2 citations
Computer Science · Decision Sciences · Mathematics · #Stochastic Gradient Optimization Techniques #Advanced Bandit Algorithms Research #Markov Chains and Monte Carlo Methods
- Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond
2023/05/22 by Itai Kreisler, Kreisler, Itai, Mor Shpigel Nacson +5 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
- Malign Overfitting: Interpolation Can Provably Preclude Invariance
2022/11/28 by Yoav Wald, Gal Yona, Wald, Yoav +5 · 1 citation
Computer Science · #Domain Adaptation and Few-Shot Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification
- A Whole New Ball Game: A Primal Accelerated Method for Matrix Games and Minimizing the Maximum of Smooth Functions
2023/11/17 by Yair Carmon, Arun Jambulapati, Carmon, Yair +5 · 1 citation
Computer Science · Decision Sciences · Mathematics · #Advanced Bandit Algorithms Research #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Markov Chains and Monte Carlo Methods #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques