2025/03/14 by Shi, Mingjia, Ruihan Lin, Lin, Ruihan +16
Computer Science · #68Q32 #Advanced Neural Network Applications #FOS: Computer and information sciences #I.2 #Machine Learning (cs.LG) #Machine Learning and Data Classification #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2503.11462
openalex publication_date 2025/03/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Learning to Optimize (L2O) enhances optimization efficiency with integrated neural networks. L2O paradigms achieve great outcomes, e.g., refitting optimizer, generating unseen solutions iteratively or directly. However, conventional L2O methods require intricate design and rely on specific optimization processes, limiting scalability and generalization. Our analyses explore general framework for learning optimization, called Diff-L2O, focusing on augmenting sampled solutions from a wider view rather than local updates in real optimization process only. Meanwhile, we give the related generalization bound, showing that the sample diversity of Diff-L2O brings better performance. This bound can be simply applied to other fields, discussing diversity, mean-variance, and different tasks. Diff-L2O's strong compatibility is empirically verified with only minute-level training, comparing with other hour-levels.