2019/06/28 by M. J. V. Streeter, Matthew Streeter, Streeter, Matthew
Computer Science · Mathematics · #Business #Computer science #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1907.00103
arxiv created 2019/06/28 · openalex publication_date 2019/06/28 · arxiv updated 2019/07/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider the problem of learning a loss function which, when minimized over a training dataset, yields a model that approximately minimizes a validation error metric. Though learning an optimal loss function is NP-hard, we present an anytime algorithm that is asymptotically optimal in the worst case, and is provably efficient in an idealized "easy" case. Experimentally, we show that this algorithm can be used to tune loss function hyperparameters orders of magnitude faster than state-of-the-art alternatives. We also show that our algorithm can be used to learn novel and effective loss functions on-the-fly during training.