2016/09/12 by Sayantan Dasgupta, Dasgupta, Sayantan
Computer Science · #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Machine Learning (cs.LG) #Natural Language Processing Techniques #Text and Document Classification Technologies
paper · pdf · doi:10.48550/arxiv.1609.03426
openalex publication_date 2016/09/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Here we study the problem of learning labels for large text corpora where each text can be assigned a variable number of labels. The problem might seem trivial when the label dimensionality is small and can be easily solved using a series of one-vs-all classifiers. However, as the label dimensionality increases to several thousand, the parameter space becomes extremely large, and it is no longer possible to use the one-vs-all technique. Here we propose a model based on the factorization of higher order moments of the words in the corpora, as well as the cross moment between the labels and the words for multi-label prediction. Our model provides guaranteed convergence bounds on the estimated parameters. Further, our model takes only three passes through the training dataset to extract the parameters, resulting in a highly scalable algorithm that can train on GB's of data consisting of millions of documents with hundreds of thousands of labels using a nominal resource of a single processor with 16GB RAM. Our model achieves 10x-15x order of speed-up on large-scale datasets while producing competitive performance in comparison with existing benchmark algorithms.