Estimation of probabilities from sparse data for the language model component of a speech recognizer

1987/03/01 by S. Katz, Slava M. Katz · 11 citations
Computer Science · #Natural Language Processing Techniques #Speech Recognition and Synthesis #Speech and Audio Processing

paper · doi:10.1109/tassp.1987.1165125

openalex publication_date 1987/03/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/22

Abstract

The description of a novel type of m-gram language model is given. The model offers, via a nonlinear recursive procedure, a computation and space efficient solution to the problem of estimating probabilities from sparse data. This solution compares favorably to other proposed methods. While the method has been developed for and successfully implemented in the IBM Real Time Speech Recognizers, its generality makes it applicable in other areas where the problem of estimating probabilities from sparse data arises.

Cited by

• Are Statistical Methods Obsolete in the Era of Deep Learning? A Study of ODE Inverse Problems
• Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
• Vibe Learning: Education in the age of AI
• Education Paradigm Shift To Maintain Human Competitive Advantage Over AI
• A language model for very large-vocabulary speech recognition
• Allocation of Parameters in Transformers
• MoPE: A Mixture of Password Experts for Improving Password Guessing
• A Winnow-Based Approach to Context-Sensitive Spelling Correction
• Variable Word Rate N-grams
• Differentiable Weighted Finite-State Transducers
• Towards the Next-generation Bayesian Network Classifiers

Related

• Analyzing and Improving Statistical Language Models for Speech Recognition
• Probabilistic top-down parsing and language modeling
• Refinement of a Structured Language Model
• Skip-gram Language Modeling Using Sparse Non-negative Matrix Probability Estimation
• Robust Probabilistic Predictive Syntactic Processing