Model Interpolation with Trans-dimensional Random Field Language Models for Speech Recognition

2016/03/30 by Bin Wang, Wang, Bin, Zhijian Ou +5 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1603.09170

openalex publication_date 2016/03/30 · openalex created_date 2016/09/16 · openalex updated_date 2026/07/28

Abstract

The dominant language models (LMs) such as n-gram and neural network (NN) models represent sentence probabilities in terms of conditionals. In contrast, a new trans-dimensional random field (TRF) LM has been recently introduced to show superior performances, where the whole sentence is modeled as a random field. In this paper, we examine how the TRF models can be interpolated with the NN models, and obtain 12.1% and 17.9% relative error rate reductions over 6-gram LMs for English and Chinese speech recognition respectively through log-linear combination.

Citations

• Recurrent Neural Network Regularization
• Feedforward Sequential Memory Networks: A New Structure to Learn Long-term Dependency

Cited by

• Integrating Discrete and Neural Features via Mixed-feature Trans-dimensional Random Field Language Models

Related

• Integrating Discrete and Neural Features via Mixed-feature Trans-dimensional Random Field Language Models
• Language modeling with Neural trans-dimensional random fields
• Learning neural trans-dimensional random field language models with noise-contrastive estimation
• Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation
• Improving N-gram Language Models with Pre-trained Deep Transformer