Generating Repetitions with Appropriate Repeated Words

2022/07/03 by Toshiki Kawamoto, Hidetaka Kamigaito, Kawamoto, Toshiki +5
Arts and Humanities · Computer Science · Health Professions · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Interpreting and Communication in Healthcare #Language, Discourse, Communication Strategies #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.2207.00929

openalex publication_date 2022/07/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A repetition is a response that repeats words in the previous speaker's utterance in a dialogue. Repetitions are essential in communication to build trust with others, as investigated in linguistic studies. In this work, we focus on repetition generation. To the best of our knowledge, this is the first neural approach to address repetition generation. We propose Weighted Label Smoothing, a smoothing method for explicitly learning which words to repeat during fine-tuning, and a repetition scoring method that can output more appropriate repetitions during decoding. We conducted automatic and human evaluations involving applying these methods to the pre-trained language model T5 for generating repetitions. The experimental results indicate that our methods outperformed baselines in both evaluations.

Related

• Taming Repetition in Dialogue Generation
• Repetition Neurons: How Do Language Models Produce Repetitions?
• Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation
• Repetitions are not all alike: distinct mechanisms sustain repetition in language models
• Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation