2016/04/20 by Lin Xu, Shaobo Lin, Xu, Lin +8
Computer Science · Engineering · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Sparse and Compressive Sensing Techniques #cs.LG
paper · pdf · doi:10.48550/arxiv.1604.05993
12 pages, 6 figures. arXiv admin note: text overlap with arXiv:1411.3553
arxiv created 2016/04/20 · openalex publication_date 2016/04/20 · arxiv updated 2016/04/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/02
Orthogonal greedy learning (OGL) is a stepwise learning scheme that starts with selecting a new atom from a specified dictionary via the steepest gradient descent (SGD) and then builds the estimator through orthogonal projection. In this paper, we find that SGD is not the unique greedy criterion and introduce a new greedy criterion, called "δ-greedy threshold" for learning. Based on the new greedy criterion, we derive an adaptive termination rule for OGL. Our theoretical study shows that the new learning scheme can achieve the existing (almost) optimal learning rate of OGL. Plenty of numerical experiments are provided to support that the new scheme can achieve almost optimal generalization performance, while requiring less computation than OGL.