Dealing with a large number of classes -- Likelihood, Discrimination or Ranking?

2016/06/22 by David Barber, Aleksandar Botev, Barber, David +1 · 1 citation
Mathematics · #Census and Population Estimation #FOS: Computer and information sciences #Machine Learning (stat.ML) #stat.ML

paper · pdf · doi:10.48550/arxiv.1606.06959

openalex publication_date 2016/06/22 · arxiv created 2016/07/07 · arxiv updated 2016/07/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider training probabilistic classifiers in the case of a large number of classes. The number of classes is assumed too large to perform exact normalisation over all classes. To account for this we consider a simple approach that directly approximates the likelihood. We show that this simple approach works well on toy problems and is competitive with recently introduced alternative non-likelihood based approximations. Furthermore, we relate this approach to a simple ranking objective. This leads us to suggest a specific setting for the optimal threshold in the ranking objective.

Citations

• BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabularies

Cited by

• EDropout: Energy-Based Dropout and Pruning of Deep Neural Networks

Related

• Extrapolating Expected Accuracies for Large Multi-Class Problems
• Candidates vs. Noises Estimation for Large Multi-Class Classification Problem
• Multiclass Classification via Class-Weighted Nearest Neighbors
• How to Formulate and Solve Statistical Recognition and Learning Problems
• Multi-Class Classification from Single-Class Data with Confidences