SpeechBrain: A General-Purpose Speech Toolkit

2021/06/08 by Mirco Ravanelli, Titouan Parcollet, Ravanelli, Mirco +39 · 44 citations
Computer Science · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2106.04624

openalex publication_date 2021/06/08 · openalex created_date 2021/06/22 · openalex updated_date 2026/07/28

Abstract

SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing pipelines. SpeechBrain achieves competitive or state-of-the-art performance in a wide range of speech benchmarks. It also provides training recipes, pretrained models, and inference scripts for popular speech datasets, as well as tutorials which allow anyone with basic Python proficiency to familiarize themselves with speech technologies.

Citations

• Kaldi Speech Recognition Toolkit
• Sequence Transduction with Recurrent Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Long Short-Term Memory
• Multiple emitter location and signal parameter estimation
• A tutorial on spectral clustering
• fairseq: A Fast, Extensible Toolkit for Sequence Modeling
• Quaternion Recurrent Neural Networks
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• NeMo: a toolkit for building AI applications using Neural Modules
• BUT System Description to VoxCeleb Speaker Recognition Challenge 2019
• Fastai: A Layered API for Deep Learning
• LibriMix: An Open-Source Dataset for Generalizable Speech Separation
• MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
• Timers and Such: A Practical Benchmark for Spoken Language Understanding with Numbers

Cited by

• Leveraging Gradient Reversal Loss and Multitask Learning for Datasets-Aware Audio Deepfake Detection
• When De-noising Hurts: A Systematic Study of Speech Enhancement Effects on Modern Medical ASR Systems
• BEST-STD2.0: Balanced and Efficient Speech Tokenizer for Spoken Term Detection
• A stylometric analysis of speaker attribution from speech transcripts
• Leveraging Language Information for Target Language Extraction
• TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edge
• STCTS: Generative Semantic Compression for Ultra-Low Bitrate Speech via Explicit Text-Prosody-Timbre Decomposition
• Group-Aware Partial Model Merging for Children's Automatic Speech Recognition
• Train Short, Infer Long: Speech-LLM Enables Zero-Shot Streamable Joint ASR and Diarization on Long Audio
• Toward Conversational Hungarian Speech Recognition: Introducing the BEA-Large and BEA-Dialogue Datasets
• Voice-Interactive Surgical Agent for Multimodal Patient Data Control
• Potential and Pitfalls of Audio as Data for Political Research: Alignment, Features, and Classification Models
• ReFESS-QI: Reference-Free Evaluation For Speech Separation With Joint Quality And Intelligibility Scoring
• FAC-FACodec: Controllable Zero-Shot Foreign Accent Conversion with Factorized Speech Codec
• Target speaker anonymization in multi-speaker recordings
• Comparison of Speech Tasks in Human Expert and Machine Detection of Parkinson's Disease
• OASIS: A Multilingual and Multimodal Dataset for Culturally Grounded Spoken Visual QA
• How I Built ASR for Endangered Languages with a Spoken Dictionary
• Provable Speech Attributes Conversion via Latent Independence
• XPPG-PCA: Reference-free automatic speech severity evaluation with principal components
• Evaluating pretrained speech embedding systems for dysarthria detection across heterogenous datasets
• Face and voice cross-modal association with learning convex feature embedding
• ClearMask: Noise-Free and Naturalness-Preserving Protection Against Voice Deepfake Attacks
• Comparator Loss: An Ordinal Contrastive Loss to Derive a Severity Score for Speech-based Health Monitoring
• Fed-PISA: Federated Voice Cloning via Personalized Identity-Style Adaptation
• A Novel Semantic Compression Approach for Ultra-low Bandwidth Voice Communication
• Data-independent Beamforming for End-to-end Multichannel Multi-speaker ASR
• Efficient Trie-based Biasing using K-step Prediction for Rare Word Recognition
• Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
• DarkStream: real-time speech anonymization with low latency
• Analysis of Speaker Verification Performance Trade-offs with Neural Audio Codec Transmission
• Digits-In-Noise Hearing Test Using Text-to-Speech and Automatic Speech Recognition: Proof-of-Concept Study
• Speaker-Conditioned Phrase Break Prediction for Text-to-Speech with Phoneme-Level Pre-trained Language Model
• FreeTalk:A plug-and-play and black-box defense against speech synthesis attacks
• Benchmarking Large Pretrained Multilingual Models on Québec French Speech Recognition
• Towards Low-Latency Tracking of Multiple Speakers With Short-Context Speaker Embeddings
• Advances in Speech Separation: Techniques, Challenges, and Future Trends
• G-IFT: A Gated Linear Unit adapter with Iterative Fine-Tuning for Low-Resource Children's Speaker Verification
• Privacy Disclosure of Similarity Rank in Speech and Language Processing
• Enhancing Dialogue Annotation with Speaker Characteristics Leveraging a Frozen LLM
• Distilling Knowledge from Ensembles of Acoustic Models for Joint CTC-Attention End-to-End Speech Recognition
• Revisiting the Privacy of Low-Frequency Speech Signals: Exploring Resampling Methods, Evaluation Scenarios, and Speaker Characteristics
• Word Error Rate Definitions and Algorithms for Long-Form Multi-talker Speech Recognition
• Reference-free Adversarial Sex Obfuscation in Speech
• MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks

Related

• Open-Source Conversational AI with SpeechBrain 1.0
• The PyTorch-Kaldi Speech Recognition Toolkit
• The 2025 PNPL Competition: Speech Detection and Phoneme Classification in the LibriBrain Dataset
• ESPnet: End-to-End Speech Processing Toolkit
• ClearerVoice-Studio: Bridging Advanced Speech Processing Research and Practical Deployment