2019/06/05 by Sypherd, Tyler, Diaz, Mario, Cava, John Kevin +3 · 3 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)
paper · doi:10.48550/arxiv.1906.02314
We introduce a tunable loss function called α-loss, parameterized by α∈ (0,∞], which interpolates between the exponential loss (α= 1/2), the log-loss (α= 1), and the 0-1 loss (α= ∞), for the machine learning setting of classification. Theoretically, we illustrate a fundamental connection between α-loss and Arimoto conditional entropy, verify the classification-calibration of α-loss in order to demonstrate asymptotic optimality via Rademacher complexity generalization techniques, and build-upon a notion called strictly local quasi-convexity in order to quantitatively characterize the optimization landscape of α-loss. Practically, we perform class imbalance, robustness, and classification experiments on benchmark image datasets using convolutional-neural-networks. Our main practical conclusion is that certain tasks may benefit from tuning α-loss away from log-loss (α= 1), and to this end we provide simple heuristics for the practitioner. In particular, navigating the α hyperparameter can readily provide superior model robustness to label flips (α> 1) and sensitivity to imbalanced classes (α< 1).