AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion

2025/05/28 by Junqi Zhao, Jinzheng Zhao, Zhao, Junqi +13 · 3 citations
Computer Science · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Generative Adversarial Networks and Image Synthesis #Music Technology and Sound Studies #Music and Audio Processing #Sound (cs.SD) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2505.22106

openalex publication_date 2025/05/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Diffusion models have significantly improved the quality and diversity of audio generation but are hindered by slow inference speed. Rectified flow enhances inference speed by learning straight-line ordinary differential equation (ODE) paths. However, this approach requires training a flow-matching model from scratch and tends to perform suboptimally, or even poorly, at low step counts. To address the limitations of rectified flow while leveraging the advantages of advanced pre-trained diffusion models, this study integrates pre-trained models with the rectified diffusion method to improve the efficiency of text-to-audio (TTA) generation. Specifically, we propose AudioTurbo, which learns first-order ODE paths from deterministic noise sample pairs generated by a pre-trained TTA model. Experiments on the AudioCaps dataset demonstrate that our model, with only 10 sampling steps, outperforms prior models and reduces inference to 3 steps compared to a flow-matching-based acceleration model.

Citations

• Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow
• SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer
• Universal Sound Separation with Self-Supervised Audio Masked Autoencoder
• LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation
• A Survey of Deep Learning Audio Generation Methods
• Leveraging AI to Generate Audio for User-generated Content in Video Games
• FlashSpeech: Efficient Zero-Shot Speech Synthesis
• Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation
• UniAudio: An Audio Foundation Model Toward Universal Audio Generation
• Matcha-TTS: A fast TTS architecture with conditional flow matching
• AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining
• Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation
• Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models
• AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
• MuChoMusic dataset
• Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation
• Flow Matching for Generative Modeling
• AudioGen: Textually Guided Audio Generation
• Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
• Classifier-Free Diffusion Guidance
• Diffsound: Discrete Diffusion Model for Text-to-sound Generation
• ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
• Pseudo Numerical Methods for Diffusion Models on Manifolds
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Diffusion Models Beat GANs on Image Synthesis
• Learning Transferable Visual Models From Natural Language Supervision
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Implicit Models
• PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition
• Neural Discrete Representation Learning
• U-Net: Convolutional Networks for Biomedical Image Segmentation

Cited by

• DreamFoley: Scalable VLMs for High-Fidelity Video-to-Audio Generation
• AudioRAG+: Feedback-driven Retrieval-augmented Audio Generation with Large Audio Language Models
• MeanAudio: Fast and Faithful Text-to-Audio Generation with Mean Flows

Related

• FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation
• LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation
• ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation
• AudioLCM: Text-to-Audio Generation with Latent Consistency Models
• VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching