Controlling Output Length in Neural Encoder-Decoders

2016/09/30 by Yuta Kikuchi, Kikuchi, Yuta, Graham Neubig +7 · 11 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1609.09552

11 pages. To appear in EMNLP 2016

arxiv created 2016/09/30 · openalex publication_date 2016/09/30 · arxiv updated 2016/10/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Neural encoder-decoder models have shown great success in many sequence generation tasks. However, previous work has not investigated situations in which we would like to control the length of encoder-decoder outputs. This capability is crucial for applications such as text summarization, in which we have to generate concise summaries with a desired length. In this paper, we propose methods for controlling the output sequence length for neural encoder-decoder models: two decoding-based methods and two learning-based methods. Results show that our learning-based methods have the capability to control length without degrading summary quality in a summarization task.

Citations

• Long Short-Term Memory
• Bidirectional recurrent neural networks
• Sequence Level Training with Recurrent Neural Networks
• Generating News Headlines with Recurrent Neural Networks
• Incorporating Copying Mechanism in Sequence-to-Sequence Learning
• A Persona-Based Neural Conversation Model
• Effective Approaches to Attention-based Neural Machine Translation

Cited by

• Progress Ratio Embeddings: An Impatience Signal for Robust Length Control in Neural Text Generation
• ConInstruct: Evaluating Large Language Models on Conflict Detection and Resolution in Instructions
• Reasoning-Guided Claim Normalization for Noisy Multilingual Social Media Posts
• Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization
• Length Representations in Large Language Models
• Controlling Summarization Length Through EOS Token Weighting
• Steering Output Style and Topic in Neural Response Generation
• Polite Dialogue Generation Without Parallel Data
• TransSent: Towards Generation of Structured Sentences with Discourse Marker
• Actor-Critic based Training Framework for Abstractive Summarization
• Positional Encoding to Control Output Sequence Length

Related

• Positional Encoding to Control Output Sequence Length
• Controllable Length Control Neural Encoder-Decoder via Reinforcement Learning
• Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization
• Closed-Book Training to Improve Summarization Encoder Memory
• Windowing Models for Abstractive Summarization of Long Texts