Qwen vs. Gemma Integration with Whisper: A Comparative Study in Multilingual SpeechLLM Systems

2025/06/16 by Tuan M. Nguyen, Nguyen, Tuan, Long-Vu Hoang +2 · 2 citations
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Multi-Agent Systems and Negotiation #Natural Language Processing Techniques #Sound (cs.SD) #Speech and dialogue systems #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2506.13596

openalex publication_date 2025/06/16 · openalex created_date 2025/10/20 · openalex updated_date 2026/07/28

Abstract

This paper presents our system for the MLC-SLM Challenge 2025, focusing on multilingual speech recognition and language modeling with large language models (LLMs). Our approach combines a fine-tuned Whisper-large-v3 encoder with efficient projector architectures and various decoder configurations. We employ a three-stage training methodology that progressively optimizes the encoder, projector, and LLM components. Our system achieves competitive performance with a private test average WER/CER result of 16.63% using the Gemma3-12B and 18.6% using the Qwen2.5-7B as decoder-only language model.

Citations

• Gemma 3 Technical Report
• Phi-4 Technical Report
• Qwen2 Technical Report
• An Embarrassingly Simple Approach for LLM with Strong ASR Capacity
• FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
• AudioPaLM: A Large Language Model That Can Speak and Listen
• LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
• Visual Instruction Tuning
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Robust Speech Recognition via Large-Scale Weak Supervision
• GLU Variants Improve Transformer

Cited by

• PART: Progressive Alignment Representation Training for Multilingual Speech-To-Text with LLMs
• Summary on The Multilingual Conversational Speech Language Model Challenge: Datasets, Tasks, Baselines, and Methods

Related

• Transsion Multilingual Speech Recognition System for MLC-SLM 2025 Challenge
• Triple X: A LLM-Based Multilingual Speech Recognition System for the INTERSPEECH2025 MLC-SLM Challenge
• TTA: Transcribe, Translate and Alignment for Cross-lingual Speech Representation
• SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge
• Speech LLMs in Low-Resource Scenarios: Data Volume Requirements and the Impact of Pretraining on High-Resource Languages