Neural Sequence Model Training via α-divergence Minimization

2017/06/30 by Sotetsu Koyamada, Koyamada, Sotetsu, Yuta Kikuchi +7
Computer Science · #Adversarial Robustness in Machine Learning #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.1706.10031

openalex publication_date 2017/06/30 · openalex created_date 2017/07/14 · openalex updated_date 2026/07/28

Abstract

We propose a new neural sequence model training method in which the objective function is defined by α-divergence. We demonstrate that the objective function generalizes the maximum-likelihood (ML)-based and reinforcement learning (RL)-based objective functions as special cases (i.e., ML corresponds to α→ 0 and RL to α→1). We also show that the gradient of the objective function can be considered a mixture of ML- and RL-based objective gradients. The experimental results of a machine translation task show that minimizing the objective function with α> 0 outperforms α→ 0, which corresponds to ML-based methods.

Citations

• A Neural Conversational Model
• An Actor-Critic Algorithm for Sequence Prediction
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Effective Approaches to Attention-based Neural Machine Translation
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Related

• Learning Neural Sequence-to-Sequence Models from Weak Feedback with\n Bipolar Ramp Loss
• Bridging Maximum Likelihood and Adversarial Learning via α-Divergence
• Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach
• MLE-guided parameter search for task loss minimization in neural sequence modeling
• Error-free Training for Artificial Neural Network