2022/01/16 by Chen Zhao, Zhao Chen, Vincent Casser +6 · 1 citation
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Topic Modeling #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.2201.05938
15 pages (including Appendix), 8 figures
openalex publication_date 2022/01/16 · arxiv created 2022/01/19 · arxiv updated 2022/01/20 · openalex created_date 2022/04/03 · openalex updated_date 2026/07/28
We propose GradTail, an algorithm that uses gradients to improve model performance on the fly in the face of long-tailed training data distributions. Unlike conventional long-tail classifiers which operate on converged - and possibly overfit - models, we demonstrate that an approach based on gradient dot product agreement can isolate long-tailed data early on during model training and improve performance by dynamically picking higher sample weights for that data. We show that such upweighting leads to model improvements for both classification and regression models, the latter of which are relatively unexplored in the long-tail literature, and that the long-tail examples found by gradient alignment are consistent with our semantic expectations.