Mixture of Low-Rank Adapter Experts in Generalizable Audio Deepfake Detection

2025/09/17 by Ivan Kukanov, Laakkonen, Janne, Ville Hautamäki +2
Computer Science · #Generative Adversarial Networks and Image Synthesis #Digital Media Forensic Detection #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.2509.13878

Abstract

Foundation models such as Wav2Vec2 excel at representation learning in speech tasks, including audio deepfake detection. However, after being fine-tuned on a fixed set of bonafide and spoofed audio clips, they often fail to generalize to novel deepfake methods not represented in training. To address this, we propose a mixture-of-LoRA-experts approach that integrates multiple low-rank adapters (LoRA) into the model's attention layers. A routing mechanism selectively activates specialized experts, enhancing adaptability to evolving deepfake attacks. Experimental results show that our method outperforms standard fine-tuning in both in-domain and out-of-domain scenarios, reducing equal error rates relative to baseline models. Notably, our best MoE-LoRA model lowers the average out-of-domain EER from 8.55% to 6.08%, demonstrating its effectiveness in achieving generalizable audio deepfake detection.

Citations

• Generalizable speech deepfake detection via meta-learned LoRA
• Meta-Learning Approaches for Improving Detection of Unseen Speech Deepfakes
• Leveraging Mixture of Experts for Improved Speech Deepfake Detection
• Malacopula: adversarial automatic speaker verification attacks using a neural-based generalised Hammerstein model
• ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale
• AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Experts
• A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI
• FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion
• Does Audio Deepfake Detection Generalize?
• Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation
• AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks
• ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection
• FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset
• LoRA: Low-Rank Adaptation of Large Language Models
• HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
• SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Experts
• Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• An explainability study of the constant Q cepstral coefficient spoofing countermeasure for automatic speaker verification
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Adaptive Mixtures of Local Experts
• The Graph Neural Network Model

Related

• MoLEx: Mixture of LoRA Experts in Speech Self-Supervised Models for Audio Deepfake Detection
• Generalizable speech deepfake detection via meta-learned LoRA
• Wav2DF-TSL: Two-stage Learning with Efficient Pre-training and Hierarchical Experts Fusion for Robust Audio Deepfake Detection
• Attention-based Mixture of Experts for Robust Speech Deepfake Detection
• Improving Out-of-Domain Audio Deepfake Detection via Layer Selection and Fusion of SSL-Based Countermeasures