2024/06/26 by S. H. Ye, Ye, Shuaishuai, Shengnan Chen +5 · 1 citation
Computer Science · #Audio and Speech Processing (eess.AS) #Blind Source Separation Techniques #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #Target Tracking and Data Fusion in Sensor Networks #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2406.18021
openalex publication_date 2024/06/26 · openalex created_date 2024/06/28 · openalex updated_date 2026/07/28
In this work, we propose a Switch-Conformer-based MoE system named SC-MoE for unified streaming and non-streaming code-switching (CS) automatic speech recognition (ASR), where we design a streaming MoE layer consisting of three language experts, which correspond to Mandarin, English, and blank, respectively, and equipped with a language identification (LID) network with a Connectionist Temporal Classification (CTC) loss as a router in the encoder of SC-MoE to achieve a real-time streaming CS ASR system. To further utilize the language information embedded in text, we also incorporate MoE layers into the decoder of SC-MoE. In addition, we introduce routers into every MoE layer of the encoder and the decoder and achieve better recognition performance. Experimental results show that the SC-MoE significantly improves CS ASR performances over baseline with comparable computational efficiency.