Ilyas Fatkhullin
- EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback
2021/06/09 by Peter Richtárik, Richtárik, Peter, Igor Sokolov +3 · 18 citations
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Privacy-Preserving Technologies in Data #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- From Gradient Clipping to Normalization for Heavy Tailed SGD
2024/10/17 by Florian Hübler, Hübler, Florian, Ilyas Fatkhullin +3 · 20 citations
Mathematics · Physics and Astronomy · Engineering · #Gas Dynamics and Kinetic Theory #Atomic and Molecular Physics #Advanced Numerical Methods in Computational Mathematics
- Momentum Provably Improves Error Feedback!
2023/05/24 by Ilyas Fatkhullin, Fatkhullin, Ilyas, Alexander Tyurin +3 · 10 citations
Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Privacy-Preserving Technologies in Data
- Sharp Analysis of Stochastic Optimization under Global Kurdyka-Łojasiewicz Inequality
2022/10/04 by Ilyas Fatkhullin, Jalal Etesami, Fatkhullin, Ilyas +5 · 5 citations
Computer Science · Decision Sciences · Engineering · #90C15 #90C26 #Advanced Bandit Algorithms Research #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods
2023/05/21 by Junchi Yang, Yang, Junchi, Xiang Li +5 · 5 citations
Computer Science · Engineering · #Advanced Image Processing Techniques #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
2021/10/07 by Ilyas Fatkhullin, Fatkhullin, Ilyas, Igor Sokolov +7 · 2 citations
Computer Science · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Numerical Methods and Algorithms #Optimization and Control (math.OC)
- 3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation
2022/02/02 by Peter Richtárik, Richtárik, Peter, Igor Sokolov +9 · 1 citation
Computer Science · Engineering · #Data Structures and Algorithms (cs.DS) #Distributed #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning and ELM #Optimization and Control (math.OC) #Parallel #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
- Stochastic Optimization under Hidden Convexity
2023/12/30 by Ilyas Fatkhullin, Fatkhullin, Ilyas, Niao He +3 · 2 citations
Computer Science · Engineering · #90C06 #90C15 #90C26 #Age of Information Optimization #Computational Complexity (cs.CC) #FOS: Computer and information sciences #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Learning Zero-Sum Linear Quadratic Games with Improved Sample Complexity and Last-Iterate Convergence
2023/09/08 by Jiduan Wu, Wu, Jiduan, Anas Barakat +5 · 1 citation
Computer Science · Decision Sciences · #Adaptive Dynamic Programming Control #Advanced Bandit Algorithms Research #Computer Science and Game Theory (cs.GT) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Systems and Control (eess.SY) #electronic engineering #information engineering