Short Text Clustering with Transformers

2021/01/31 by Leonid Pugachev, Pugachev, Leonid, Mikhail Burtsev +1
Computer Science · #Natural Language Processing Techniques #Text and Document Classification Technologies #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2102.00541

arxiv created 2021/01/31 · arxiv updated 2021/02/02

Abstract

Recent techniques for the task of short text clustering often rely on word embeddings as a transfer learning component. This paper shows that sentence vector representations from Transformers in conjunction with different clustering methods can be successfully applied to address the task. Furthermore, we demonstrate that the algorithm of enhancement of clustering via iterative classification can further improve initial clustering performance with different classifiers, including those based on pre-trained Transformer language models.

Related

• TWIST: Training-free and Label-free Short Text Clustering through Iterative Vector Updating with LLMs
• Semi-supervised Clustering for Short Text via Deep Representation Learning
• Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets
• Enhancement of Short Text Clustering by Iterative Classification
• Representation Learning for Short Text Clustering