LenAtten: An Effective Length Controlling Unit For Text Summarization

2021/06/01 by Zhongyi Yu, Yu, Zhongyi, Zhenghao Wu +9 · 3 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Advanced Text Analysis Techniques

paper · pdf · doi:10.48550/arxiv.2106.00316

Abstract

Fixed length summarization aims at generating summaries with a preset number of words or characters. Most recent researches incorporate length information with word embeddings as the input to the recurrent decoding unit, causing a compromise between length controllability and summary quality. In this work, we present an effective length controlling unit Length Attention (LenAtten) to break this trade-off. Experimental results show that LenAtten not only brings improvements in length controllability and ROGUE scores but also has great generalization ability. In the task of generating a summary with the target length, our model is 732 times better than the best-performing length controllable summarizer in length controllability on the CNN/Daily Mail dataset.

Cited by

• Progress Ratio Embeddings: An Impatience Signal for Robust Length Control in Neural Text Generation
• Prompt-Based One-Shot Exact Length-Controlled Generation with LLMs
• Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization

Related

• Summarization with Precise Length Control
• Length-controllable Abstractive Summarization by Guiding with Summary Prototype
• Zero-Shot Strategies for Length-Controllable Summarization
• A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarization
• Reinforced Abstractive Summarization with Adaptive Length Controlling