Laughter Synthesis: Combining Seq2seq modeling with Transfer Learning

2020/08/20 by Tits, Noé, Haddad, Kevin El, Dutoit, Thierry
#Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2008.09483

Abstract

Despite the growing interest for expressive speech synthesis, synthesis of nonverbal expressions is an under-explored area. In this paper we propose an audio laughter synthesis system based on a sequence-to-sequence TTS synthesis system. We leverage transfer learning by training a deep learning model to learn to generate both speech and laughs from annotations. We evaluate our model with a listening test, comparing its performance to an HMM-based laughter synthesis one and assess that it reaches higher perceived naturalness. Our solution is a first step towards a TTS system that would be able to synthesize speech with a control on amusement level with laughter integration.

Related

• LaughNet: synthesizing laughter utterances from waveform silhouettes and a single laughter example
• Laughter Synthesis using Pseudo Phonetic Tokens with a Large-scale In-the-wild Laughter Corpus
• Making Flow-Matching-Based Zero-Shot Text-to-Speech Laugh as You Like
• Laughing Matters: Introducing Laughing-Face Generation using Diffusion Models
• Generating Diverse Realistic Laughter for Interactive Art