2019/09/14 by Khe Chai Sim, Sim, Khe Chai, Petr Zadrazil +3 · 1 citation
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.1909.06678
openalex publication_date 2019/09/14 · openalex created_date 2022/07/19 · openalex updated_date 2026/07/28
Speaker-independent speech recognition systems trained with data from many\nusers are generally robust against speaker variability and work well for a\nlarge population of speakers. However, these systems do not always generalize\nwell for users with very different speech characteristics. This issue can be\naddressed by building personalized systems that are designed to work well for\neach specific user. In this paper, we investigate the idea of securely training\npersonalized end-to-end speech recognition models on mobile devices so that\nuser data and models never leave the device and are never stored on a server.\nWe study how the mobile training environment impacts performance by simulating\non-device data consumption. We conduct experiments using data collected from\nspeech impaired users for personalization. Our results show that\npersonalization achieved 63.7 % relative word error rate reduction when trained\nin a server environment and 58.1% in a mobile environment. Moving to on-device\npersonalization resulted in 18.7% performance degradation, in exchange for\nimproved scalability and data privacy. To train the model on device, we split\nthe gradient computation into two and achieved 45% memory reduction at the\nexpense of 42% increase in training time.\n