Sujay Sanghavi

1. DataComp-LM: In search of the next generation of training sets for language models
   2024/06/17 by Jeffrey Li, Alex Chengyu Fang, Li, Jeffrey +121 · 2 voices · 80 citations
   Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.LG
2. Time Weaver: A Conditional Time Series Generation Model
   2024/03/05 by Sai Shankar Narasimhan, Shubhankar Agarwal, Narasimhan, Sai Shankar +7 · 9 citations
   Computer Science · #Time Series Analysis and Forecasting #Neural Networks and Applications
3. Non-convex Robust PCA
   2014/10/28 by Praneeth Netrapalli, Netrapalli, Praneeth, U. N. Niranjan +7 · 3 citations
   Computer Science · Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
4. Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling
   2018/06/26 by Shanshan Wu, Alexandros G. Dimakis, Wu, Shanshan +13 · 4 citations
   Engineering · #CCD and CMOS Imaging Sensors #Electrical and Bioimpedance Tomography #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sparse and Compressive Sensing Techniques
5. Early Weight Averaging meets High Learning Rates for LLM Pre-training
   2023/06/05 by Sunny Sanyal, Sanyal, Sunny, Jean Kaddour +6 · 5 citations
   Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling
6. Positive Unlabeled Contrastive Learning
   2022/06/01 by Anish Acharya, Acharya, Anish, Sujay Sanghavi +11 · 4 citations
   Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Infrastructure Maintenance and Monitoring #Machine Learning (cs.LG) #Machine Learning and Data Classification
7. In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
   2024/02/18 by Liam Collins, Collins, Liam, Advait Parulekar +7 · 6 citations
   Computer Science · Neuroscience · #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neural dynamics and brain function
8. Completing Any Low-rank Matrix, Provably
   2013/06/12 by Yudong Chen, Chen, Yudong, Srinadh Bhojanapalli +5 · 2 citations
   Computer Science · Engineering · #Blind Source Separation Techniques #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Microwave Imaging and Scattering Analysis #Sparse and Compressive Sensing Techniques
9. Gossiping with Multiple Messages
   2006/12/22 by Sujay Sanghavi, Sanghavi, Sujay, Bruce Hajek +3 · 1 citation
   Computer Science · #Caching and Content Delivery #FOS: Computer and information sciences #Information Theory (cs.IT) #Networking and Internet Architecture (cs.NI) #Opportunistic and Delay-Tolerant Networks #Peer-to-Peer Network Technologies
10. Understanding Self-Supervised Learning via Gaussian Mixture Models
    2024/11/05 by Parikshit Bansal, Bansal, Parikshit, Ali Kavis +3 · 4 citations
    Computer Science · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG)
11. Towards Quantifying the Preconditioning Effect of Adam
    2024/02/11 by Rudrajit Das, Das, Rudrajit, Naman Agarwal +5 · 2 citations
    Materials Science · #FOS: Computer and information sciences #FOS: Mathematics #High voltage insulation and dielectric phenomena #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Numerical Analysis (math.NA) #Optimization and Control (math.OC)
12. When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models
    2024/04/12 by Sunny Sanyal, Sanyal, Sunny, Ravid Shwartz-Ziv +5 · 2 voices · 1 citation
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.AI #cs.CL #cs.LG
13. On the Convergence of Differentially Private Federated Learning on Non-Lipschitz Objectives, and with Normalized Client Updates
    2021/06/13 by Rudrajit Das, Das, Rudrajit, Abolfazl Hashemi +5 · 1 citation
    Computer Science · #Cryptography and Data Security #Distributed #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Parallel #Privacy-Preserving Technologies in Data #Signal Processing (eess.SP) #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC) #electronic engineering #information engineering
14. Robust Training in High Dimensions via Block Coordinate Geometric Median Descent
    2021/06/16 by Anish Acharya, Acharya, Anish, Abolfazl Hashemi +9 · 1 citation
    Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Machine Learning and Algorithms #Sparse and Compressive Sensing Techniques
15. Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
    2025/02/05 by Sunny Sanyal, Hayden Prairie, Sanyal, Sunny +7 · 2 citations
    Computer Science · #Domain Adaptation and Few-Shot Learning
16. Understanding Self-Distillation in the Presence of Label Noise
    2023/01/30 by Rudrajit Das, Das, Rudrajit, Sujay Sanghavi +1 · 1 citation
    Computer Science · #Machine Learning and Data Classification #Anomaly Detection Techniques and Applications #Neural Networks and Applications
17. Understanding the Training Speedup from Sampling with Approximate Losses
    2024/02/10 by Rudrajit Das, Das, Rudrajit, Xi Chen +7 · 1 citation
    Computer Science · Engineering · #Control Systems and Identification #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms
18. Latent Variable Representation for Reinforcement Learning
    2022/12/17 by Tongzheng Ren, Ren, Tongzheng, Chenjun Xiao +13 · 1 citation
    Computer Science · #Reinforcement Learning in Robotics