Disentangling Score Content and Performance Style for Joint Piano Rendering and Transcription

2025/09/28 by Zeng, Wei, Zhao, Junchuan, Wang, Ye
#Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Multimedia (cs.MM) #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2509.23878

Abstract

Expressive performance rendering (EPR) and automatic piano transcription (APT) are fundamental yet inverse tasks in music information retrieval: EPR generates expressive performances from symbolic scores, while APT recovers scores from performances. Despite their dual nature, prior work has addressed them independently. In this paper we propose a unified framework that jointly models EPR and APT by disentangling note-level score content and global performance style representations from both paired and unpaired data. Our framework is built on a transformer-based sequence-to-sequence architecture and is trained using only sequence-aligned data, without requiring fine-grained note-level alignment. To automate the rendering process while ensuring stylistic compatibility with the score, we introduce an independent diffusion-based performance style recommendation module that generates style embeddings directly from score content. This modular component supports both style transfer and flexible rendering across a range of expressive styles. Experimental results from both objective and subjective evaluations demonstrate that our framework achieves competitive performance on EPR and APT tasks, while enabling effective content-style disentanglement, reliable style transfer, and stylistically appropriate rendering. Demos are available at https://jointpianist.github.io/epr-apt/

Citations

• End-to-end Piano Performance-MIDI to Score Conversion with Transformers
• DExter: Learning and Controlling Performance Expression with Diffusion Models
• End-to-End Real-World Polyphonic Piano Audio-to-Score Transcription with Hierarchical Decoding
• miditok: A Python package for MIDI file tokenization
• Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling
• Automatic Piano Transcription with Hierarchical Frequency-Time Transformer
• Reconstructing Human Expressiveness in Piano Performances with a Transformer Network
• The ACCompanion: Combining Reactivity, Robustness, and Musical Expressivity in an Automatic Piano Accompanist
• GPT-4 Technical Report
• Posterior Collapse and Latent Variable Non-identifiability
• Disentangled Representation Learning
• Sketching the Expression: Flexible Rendering of Expressive Piano Performance with Self-Supervised Learning
• Towards Disentangled Speech Representations
• Masked Autoencoders Are Scalable Vision Learners
• Sequence-to-Sequence Piano Transcription with Transformers
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Denoising Diffusion Probabilistic Models
• Improving Disentangled Text Representation Learning with Information-Theoretic Guidance
• Language Models are Few-Shot Learners
• GLU Variants Improve Transformer
• Analyzing and Improving the Image Quality of StyleGAN
• A holistic approach to polyphonic music transcription with neural\n networks
• Generating Sentences from Disentangled Syntactic and Semantic Spaces
• Adversarial Learning for Improved Onsets and Frames Music Transcription
• Deep Music Analogy Via Latent Representation Disentanglement
• Almost Unsupervised Text to Speech and Automatic Speech Recognition
• Learning Disentangled Joint Continuous and Discrete Representations
• Onsets and Frames: Dual-Objective Piano Transcription
• Attention Is All You Need
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Peransformer: Improving Low-informed Expressive Performance Rendering with Score-aware Discriminator
• End-to-End Real-World Polyphonic Piano Audio-to-Score Transcription with Hierarchical Decoding
• Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training
• Expressivity-aware Music Performance Retrieval using Mid-level Perceptual Features and Emotion Word Embeddings
• End-to-end Piano Performance-MIDI to Score Conversion with Transformers