2025/08/17 by Gal Lifshitz, Lifshitz, Gal, Shahar Zuler +5
Computer Science · Decision Sciences · Engineering · #Computer Vision and Pattern Recognition (cs.CV) #Educational Robotics and Engineering #FOS: Computer and information sciences #Machine Learning (cs.LG) #Nuclear reactor physics and engineering #Probabilistic and Robust Engineering Design
paper · pdf · doi:10.48550/arxiv.2508.12270
openalex publication_date 2025/08/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
End-to-end deep learning has achieved impressive results but often relies on large labeled datasets, exhibits limited generalization to unseen scenarios, and incurs substantial computational cost. Classical optimization methods, in contrast, are more data-efficient and lightweight but frequently suffer from slow convergence. Learned optimizers aim to bridge this gap, yet existing approaches have focused primarily on first-order methods, while learned second-order optimization has received much less attention. We introduce L-SR1, a learned second-order optimizer inspired by the classical Symmetric Rank-One (SR1) method. At its core, L-SR1 employs a Projection-Guided Secant Mechanism (PGSM) that generates positive semi-definite preconditioners and biases meta-training toward the quasi-Newton secant relation. Through controlled analytic benchmarks, we study stability, generalization across problem dimensions, and search direction quality, and further evaluate L-SR1 on Monocular Human Mesh Recovery (HMR), where it outperforms both classical and learned optimization-based baselines. With a compact model and no reliance on task-specific fine-tuning or annotated data, L-SR1 demonstrates strong generalization and can be integrated into a broad range of iterative optimization problems to accelerate convergence and reduce the required number of iterations.