Mark Schmidt

1. Fast Convergence of Stochastic Gradient Descent under a Strong Growth Condition
   2013/08/29 by Mark Schmidt, Schmidt, Mark, Nicolas Le Roux +1 · 19 citations
   Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Complexity and Algorithms in Graphs
2. Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
   2019/05/23 by Sharan Vaswani, Aaron Mishkin, Vaswani, Sharan +9 · 18 citations
   Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Domain Adaptation and Few-Shot Learning #Sparse and Compressive Sensing Techniques
3. Block-Coordinate Frank-Wolfe Optimization for Structural SVMs
   2012/07/19 by Simon Lacoste-Julien, Martin Jaggi, Lacoste-Julien, Simon +5 · 5 citations
   Computer Science · Engineering · #68T05 #90C06 #90C52 #90C90 #FOS: Computer and information sciences #FOS: Mathematics #G.1.6 #I.2.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
4. Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
   2023/04/27 by Frederik Künstner, Kunstner, Frederik, Jacques Chen +5 · 10 citations
   Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Metaheuristic Optimization Algorithms Research #Neural Networks and Applications #Optimization and Control (math.OC)
5. Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
   2024/02/29 by Frederik Künstner, Kunstner, Frederik, Robin Yadav +7 · 12 citations
   Computer Science · Social Sciences · #Authorship Attribution and Profiling #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #FOS: Mathematics #Language and cultural evolution #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC)
6. Robust Asymmetric Learning in POMDPs
   2020/12/31 by Andrew Warrington, Warrington, Andrew, Jonathan Lavington +7 · 4 citations
   Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and ELM #Neural Networks and Applications
7. Let's Make Block Coordinate Descent Converge Faster: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
   2017/12/23 by Julie Nutini, Nutini, Julie, Issam Laradji +3 · 3 citations
   Computer Science · Engineering · Mathematics · #90C06 #Advanced Optimization Algorithms Research #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
8. Where are the Blobs: Counting by Localization with Point Supervision
   2018/07/25 by Issam Laradji, Laradji, Issam H., Negar Rostamzadeh +7 · 3 citations
   Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Video Surveillance and Tracking Methods
9. Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data
   2025/02/07 by Fan Chen, Mark Schmidt, Fan, Chen +3 · 9 citations
   Computer Science · Business, Management and Accounting · #Face and Expression Recognition #Customer churn and segmentation #Imbalanced Data Classification Techniques
10. BlockLLM: Memory-Efficient Adaptation of LLMs by Selecting and Optimizing the Right Coordinate Blocks
    2024/06/25 by Amrutha Varshini Ramesh, Vignesh Ganapathiraman, Ramesh, Amrutha Varshini +5 · 5 citations
    Computer Science · #Neural Networks and Applications #Advanced Data Storage Technologies
11. Handling the Positive-Definite Constraint in the Bayesian Learning Rule
    2020/02/24 by Lin Wu, Mark Schmidt, Lin, Wu +3 · 3 citations
    Computer Science · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms
12. Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models
    2023/06/22 by Leonardo Galli, Galli, Leonardo, Holger Rauhut +3 · 2 citations
    Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Image and Video Retrieval Techniques #Diffusion and Search Dynamics #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Metaheuristic Optimization Algorithms Research #Optimization and Control (math.OC)
13. Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
    2023/02/20 by Lin Wu, Valentin Duruisseaux, Lin, Wu +9 · 2 citations
    Engineering · Computer Science · #3D Shape Modeling and Analysis #Advanced Numerical Analysis Techniques #Face recognition and analysis
14. Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation
    2019/10/10 by Si Yi Meng, Meng, Si Yi, Sharan Vaswani +7 · 1 citation
    Computer Science · Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
15. Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
    2023/06/05 by Frederik Künstner, Kunstner, Frederik, Victor Sanches Portella +4 · 1 citation
    Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning and Algorithms #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
16. BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization
    2023/05/30 by Fan Chen, Gaspard Choné-Ducasse, Fan, Chen +5 · 1 citation
    Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Machine Learning and ELM
17. Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields
    2015/04/16 by Mark Schmidt, Reza Babanezhad, Schmidt, Mark +9 · 2 citations
    Computer Science · #Stochastic Gradient Optimization Techniques #Domain Adaptation and Few-Shot Learning #Gaussian Processes and Bayesian Inference
18. Don't Be So Positive: Negative Step Sizes in Second-Order Methods
    2024/11/18 by Betty Shea, Shea, Betty, Mark Schmidt +1 · 1 voice · 1 citation
    Computer Science · #Neural Networks and Applications #Evolutionary Algorithms and Applications #Stochastic Gradient Optimization Techniques
19. Greedy Newton: Newton's Method with Exact Line Search
    2024/01/10 by Betty Shea, Mark Schmidt, Shea, Betty +1 · 1 citation
    Mathematics · Social Sciences · #Iterative Methods for Nonlinear Equations #Advanced Optimization Algorithms Research #Innovations in Educational Methods