Generation, Implementation and Appraisal of an N-gram based Stemming Algorithm

2013/12/17 by B. P. Pande, Bhagwati Prasad Pande, Pande, B. P. +5
Computer Science · #Algorithms and Data Compression #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.IR

paper · pdf · doi:10.48550/arxiv.1312.4824

10 pages

openalex publication_date 2013/12/17 · arxiv created 2014/01/16 · arxiv updated 2014/01/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A language independent stemmer has always been looked for. Single N-gram tokenization technique works well, however, it often generates stems that start with intermediate characters, rather than initial ones. We present a novel technique that takes the concept of N gram stemming one step ahead and compare our method with an established algorithm in the field, Porter's Stemmer. Results indicate that our N gram stemmer is not inferior to Porter's linguistic stemmer.

Related

• N-Grammer: Augmenting Transformers with latent n-grams
• N-gram Is Back: Residual Learning of Neural Text Generation with n-gram Language Model
• Character n-gram Embeddings to Improve RNN Language Models
• Splintering Nonconcatenative Languages for Better Tokenization
• Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens