2019/10/01 by Shahram Ghorbani, Ghorbani, Shahram, Soheil Khorram +2 · 1 citation
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Music and Audio Processing #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.1910.00565
openalex publication_date 2019/10/01 · openalex created_date 2020/07/23 · openalex updated_date 2026/07/28
Training acoustic models with sequentially incoming data -- while both\nleveraging new data and avoiding the forgetting effect-- is an essential\nobstacle to achieving human intelligence level in speech recognition. An\nobvious approach to leverage data from a new domain (e.g., new accented speech)\nis to first generate a comprehensive dataset of all domains, by combining all\navailable data, and then use this dataset to retrain the acoustic models.\nHowever, as the amount of training data grows, storing and retraining on such a\nlarge-scale dataset becomes practically impossible. To deal with this problem,\nin this study, we study several domain expansion techniques which exploit only\nthe data of the new domain to build a stronger model for all domains. These\ntechniques are aimed at learning the new domain with a minimal forgetting\neffect (i.e., they maintain original model performance). These techniques\nmodify the adaptation procedure by imposing new constraints including (1)\nweight constraint adaptation (WCA): keeping the model parameters close to the\noriginal model parameters; (2) elastic weight consolidation (EWC): slowing down\ntraining for parameters that are important for previously established domains;\n(3) soft KL-divergence (SKLD): restricting the KL-divergence between the\noriginal and the adapted model output distributions; and (4) hybrid SKLD-EWC:\nincorporating both SKLD and EWC constraints. We evaluate these techniques in an\naccent adaptation task in which we adapt a deep neural network (DNN) acoustic\nmodel trained with native English to three different English accents:\nAustralian, Hispanic, and Indian. The experimental results show that SKLD\nsignificantly outperforms EWC, and EWC works better than WCA. The hybrid\nSKLD-EWC technique results in the best overall performance.\n