Versatile Symbolic Music-for-Music Modeling via Function Alignment

2025/06/18 by Junyan Jiang, Jiang, Junyan, Daniel Chin +7 · 1 citation
Computer Science · Neuroscience · #FOS: Computer and information sciences #Music Technology and Sound Studies #Music and Audio Processing #Neuroscience and Music Perception #Sound (cs.SD)

paper · pdf · doi:10.48550/arxiv.2506.15548

openalex publication_date 2025/06/18 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

Many music AI models learn a map between music content and human-defined labels. However, many annotations, such as chords, can be naturally expressed within the music modality itself, e.g., as sequences of symbolic notes. This observation enables both understanding tasks (e.g., chord recognition) and conditional generation tasks (e.g., chord-conditioned melody generation) to be unified under a music-for-music sequence modeling paradigm. In this work, we propose parameter-efficient solutions for a variety of symbolic music-for-music tasks. The high-level idea is that (1) we utilize a pretrained Language Model (LM) for both the reference and the target sequence and (2) we link these two LMs via a lightweight adapter. Experiments show that our method achieves superior performance among different tasks such as chord recognition, melody generation, and drum track generation. All demos, code and model weights are publicly available.

Citations

• Function Alignment: A New Theory of Mind and Intelligence, Part I: Foundations
• InspireMusic: Integrating Super Resolution and Large Language Model for High-Fidelity Long-Form Music Generation
• Editing Music with Melody and Text: Using ControlNet for Diffusion Transformer
• SymPAC: Scalable Symbolic Music Generation With Prompts And Constraints
• Generating High-quality Symbolic Music Using Fine-grained Discriminators
• Audio Prompt Adapter: Unleashing Music Editing Abilities for Text-to-Music with Lightweight Finetuning
• MusiConGen: Rhythm and Chord Control for Transformer-Based Text-to-Music Generation
• Composer's Assistant 2: Interactive Multi-Track MIDI Infilling with Fine-Grained User Control
• MuseBarControl: Enhancing Fine-Grained Control in Symbolic Music Generation through Pre-Training and Counterfactual Loss
• MelodyT5: A Unified Score-to-Score Transformer for Symbolic Music Processing
• Joint Audio and Symbolic Conditioning for Temporally Controlled Text-to-Music Generation
• Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities
• Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language Models via Instruction Tuning
• Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models
• MuPT: A Generative Symbolic Music Pretrained Transformer
• ChatMusician: Understanding and Generating Music Intrinsically with LLM
• Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controls
• LLM Augmented LLMs: Expanding Capabilities through Composition
• Music ControlNet: Multiple Time-varying Controls for Music Generation
• Content-based Controls For Music Large Language Modeling
• MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning
• MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies
• VampNet: Music Generation via Masked Acoustic Token Modeling
• Simple and Controllable Music Generation
• MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training
• MuseCoco: Generating Symbolic Music from Text
• Efficient Neural Music Generation
• LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
• LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
• MuChoMusic dataset
• Melody transcription via generative pre-training
• Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer
• LoRA: Low-Rank Adaptation of Large Language Models
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs
• Jukebox: A Generative Model for Music
• Automatic Melody Harmonization with Triad Chords: A Comparative Study
• Music Transformer
• Attention Is All You Need
• METEOR: Melody-aware Texture-controllable Symbolic Orchestral Music Generation via Transformer VAE
• Perplexity from PLM Is Unreliable for Evaluating Text Quality
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Prefix-Tuning: Optimizing Continuous Prompts for Generation

Cited by

• PhraseVAE and PhraseLDM: Latent Diffusion for Full-Song Multitrack Symbolic Music Generation

Related

• Large Language Models' Internal Perception of Symbolic Music
• Text2midi-InferAlign: Improving Symbolic Music Generation with Inference-Time Alignment
• MMT-BERT: Chord-aware Symbolic Music Generation Based on Multitrack Music Transformer and MusicBERT
• Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization
• Exploring the Efficacy of Pre-trained Checkpoints in Text-to-Music Generation Task