Loshchilov, Ilya

1. SGDR: Stochastic Gradient Descent with Warm Restarts
   2016/08/13 by Ilya Loshchilov, Loshchilov, Ilya, Frank Hutter +1 · 533 citations
   Computer Science · Engineering · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques
2. Decoupled Weight Decay Regularization
   2017/11/14 by Ilya Loshchilov, Loshchilov, Ilya, Frank Hutter +1 · 122 citations
   Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques
3. nGPT: Normalized Transformer with Representation Learning on the Hypersphere
   2024/10/01 by Ilya Loshchilov, Cheng-Ping Hsieh, Loshchilov, Ilya +5 · 4 voices · 19 citations
   #cs.LG #cs.AI
4. A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
   2017/07/27 by Patryk Chrabąszcz, Chrabaszcz, Patryk, Ilya Loshchilov +3 · 30 citations
   Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification
5. Online Batch Selection for Faster Training of Neural Networks
   2015/11/19 by Ilya Loshchilov, Frank Hutter, Loshchilov, Ilya +1 · 25 citations
   Computer Science · #Stochastic Gradient Optimization Techniques #Advanced Neural Network Applications #Machine Learning and Data Classification
6. CMA-ES for Hyperparameter Optimization of Deep Neural Networks
   2016/04/25 by Ilya Loshchilov, Loshchilov, Ilya, Frank Hutter +1 · 12 citations
   Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine Learning and Data Classification #Neural and Evolutionary Computing (cs.NE)
7. A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization
   2014/04/21 by Loshchilov, Ilya · 2 citations
   #FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE)
8. Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari
   2018/02/24 by Chrabaszcz, Patryk, Loshchilov, Ilya, Hutter, Frank · 2 citations
   #FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE)
9. NVIDIA Nemotron 3: Efficient and Open Intelligence
   2025/12/24 by NVIDIA, :, Blakeman, Aaron +356 · 4 citations
   #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)
10. Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
    2025/12/23 by NVIDIA, :, Aaron Blakeman +623 · 1 voice · 2 citations
    #cs.CL #cs.AI #cs.LG