Suvrit Sra
- Why gradient clipping accelerates training: A theoretical justification for adaptivity
2019/05/28 by Jingzhao Zhang, Tianxing He, Zhang, Jingzhao +5 · 41 citations
Computer Science · Engineering · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Contrastive Learning with Hard Negative Samples
2020/10/09 by Joshua Robinson, Robinson, Joshua, Ching-Yao Chuang +5 · 44 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Music and Audio Processing #Human Pose and Action Recognition
- Stochastic Variance Reduction for Nonconvex Optimization
2016/03/19 by Sashank J. Reddi, Ahmed Hefny, Reddi, Sashank J. +7 · 32 citations
Computer Science · Engineering · Mathematics · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Markov Chains and Monte Carlo Methods
- Sign and Basis Invariant Networks for Spectral Graph Representation Learning
2022/02/25 by Derek Lim, Lim, Derek, Joshua A. Robinson +11 · 16 citations
Computer Science · Materials Science · #Advanced Graph Neural Networks #Computational Drug Discovery Methods #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Materials Science
- First-order Methods for Geodesically Convex Optimization
2016/02/19 by Hongyi Zhang, Zhang, Hongyi, Suvrit Sra +1 · 9 citations
Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Point processes and geometric inequalities #Sparse and Compressive Sensing Techniques
- Linear attention is (maybe) all you need (to understand transformer optimization)
2023/10/02 by Kwangjun Ahn, Xiang Cheng, Ahn, Kwangjun +9 · 18 citations
Engineering · #Artificial Intelligence (cs.AI) #Energy Load and Power Forecasting #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC)
- Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context
2023/12/11 by Xiang Cheng, Yuxin Chen, Cheng, Xiang +3 · 12 citations
Computer Science · #Neural Networks and Applications #Stochastic Gradient Optimization Techniques #Machine Learning and ELM
- Trees to Flows and Back: Unifying Decision Trees and Diffusion Models
2026/05/01 by Sai Niranjan Ramachandran, Suvrit Sra · 5 voices
#cs.LG #cond-mat.stat-mech #cs.AI
- Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity
2018/10/17 by Chulhee Yun, Suvrit Sra, Yun, Chulhee +3 · 6 citations
Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Advanced Memory and Neural Computing
- Learning Adversarial MDPs with Bandit Feedback and Unknown Transition
2019/12/03 by Chi Jin, Jin, Chi, Tiancheng Jin +7 · 7 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #I.2.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Reinforcement Learning in Robotics
- Understanding the unstable convergence of gradient descent
2022/04/03 by Kwangjun Ahn, Ahn, Kwangjun, Jingzhao Zhang +3 · 5 citations
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Machine Learning and Algorithms #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
- The Crucial Role of Normalization in Sharpness-Aware Minimization
2023/05/24 by Yan Dai, Dai, Yan, Kwangjun Ahn +3 · 6 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Advanced Neural Network Applications #Machine Learning and ELM
- Riemannian Dictionary Learning and Sparse Coding for Positive Definite\n Matrices
2015/07/09 by Anoop Cherian, Cherian, Anoop, Suvrit Sra +1 · 3 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Face and Expression Recognition #Image Retrieval and Classification Techniques
- Direct Runge-Kutta Discretization Achieves Acceleration
2018/05/01 by Jingzhao Zhang, Zhang, Jingzhao, Aryan Mokhtari +5 · 6 citations
Computer Science · Mathematics · Decision Sciences · #Stochastic Gradient Optimization Techniques #Advanced Optimization Algorithms Research #Advanced Bandit Algorithms Research
- Provably Efficient Algorithms for Multi-Objective Competitive RL
2021/02/05 by Tiancheng Yu, Yu, Tiancheng, Yi Tian +5 · 3 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Game Theory and Applications #Machine Learning (cs.LG) #Reinforcement Learning in Robotics
- An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic\n Riemannian Optimization
2017/06/10 by Reshad Hosseini, Suvrit Sra, Hosseini, Reshad +1 · 2 citations
Computer Science · Mathematics · Physics and Astronomy · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Statistical Mechanics and Entropy
- How to escape sharp minima with random perturbations
2023/05/25 by Kwangjun Ahn, Ali Jadbabaie, Ahn, Kwangjun +3 · 3 citations
Computer Science · #Artificial Intelligence (cs.AI) #Complexity and Algorithms in Graphs #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning and Algorithms #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
- AdaDelay: Delay Adaptive Distributed Stochastic Convex Optimization
2015/08/20 by Suvrit Sra, Adams Wei Yu, Sra, Suvrit +5 · 2 citations
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Privacy-Preserving Technologies in Data #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Riemannian Optimization via Frank-Wolfe Methods
2017/10/30 by Melanie Weber, Weber, Melanie, Suvrit Sra +1 · 2 citations
Computer Science · Mathematics · Medicine · #15A24 #46N10 #49Q99 #65K10 #Advanced Neuroimaging Techniques and Applications #FOS: Computer and information sciences #FOS: Mathematics #Functional Analysis (math.FA) #Machine Learning (cs.LG) #Morphological variations and asymmetry #Optimization and Control (math.OC) #Topological and Geometric Data Analysis
- Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond
2021/10/20 by Chulhee Yun, Yun, Chulhee, Shashank Rajput +3 · 2 citations
Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Privacy-Preserving Technologies in Data
- Sion's Minimax Theorem in Geodesic Metric Spaces and a Riemannian Extragradient Algorithm
2022/02/13 by Peiyuan Zhang, Zhang, Peiyuan, Jingzhao Zhang +3 · 2 citations
Computer Science · Mathematics · #Optimization and Variational Analysis #Advanced Optimization Algorithms Research #Point processes and geometric inequalities
- Efficient Sampling on Riemannian Manifolds via Langevin MCMC
2024/02/15 by Xiang Cheng, Cheng, Xiang, Jingzhao Zhang +3 · 3 citations
Computer Science · Mathematics · #Computation (stat.CO) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Probability (math.PR) #Statistical Methods and Inference #Statistics Theory (math.ST) #Topological and Geometric Data Analysis
- First-Order Methods for Linearly Constrained Bilevel Optimization
2024/06/18 by Guy Kornowski, Kornowski, Guy, Swati Padmanabhan +7 · 4 citations
Mathematics · Economics, Econometrics and Finance · Computer Science · #Advanced Optimization Algorithms Research #Stochastic processes and financial applications #Optimization and Variational Analysis
- New concavity and convexity results for symmetric polynomials and their\n ratios
2018/03/27 by Suvrit Sra, Sra, Suvrit · 1 citation
Mathematics · #Analytic and geometric function theory #Classical Analysis and ODEs (math.CA) #FOS: Mathematics #Mathematical Inequalities and Applications #Mathematical functions and polynomials #Optimization and Control (math.OC)
- Escaping Saddle Points with Adaptive Gradient Methods
2019/01/26 by Matthew Staib, Sashank J. Reddi, Staib, Matthew +7 · 1 citation
Computer Science · Engineering · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Strength from Weakness: Fast Learning Using Weak Supervision
2020/02/19 by Joshua Robinson, Stefanie Jegelka, Robinson, Joshua +3 · 1 citation
Computer Science · Engineering · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Sparse and Compressive Sensing Techniques
- Three Operator Splitting with a Nonconvex Loss Function
2021/03/08 by Alp Yurtsever, Varun Mangalick, Yurtsever, Alp +3 · 1 citation
Computer Science · Engineering · Mathematics · #FOS: Mathematics #Optimization and Control (math.OC) #Point processes and geometric inequalities #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Parallel and Distributed Block-Coordinate Frank-Wolfe Algorithms
2014/09/22 by Yu-Xiang Wang, Veeranjaneyulu Sadhanala, Wang, Yu-Xiang +9 · 1 citation
Computer Science · #Complexity and Algorithms in Graphs #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Optimization and Search Problems #Stochastic Gradient Optimization Techniques
- Revisiting Frank-Wolfe for Structured Nonconvex Optimization
2025/03/11 by Hoomaan Maskan, Maskan, Hoomaan, Y. R. Hou +5 · 2 citations
Mathematics · #Advanced Optimization Algorithms Research