Modeling Musical Structure with Artificial Neural Networks

2020/01/06 by Stefan Lattner, Lattner, Stefan
Computer Science · Neuroscience · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Multimedia (cs.MM) #Music Technology and Sound Studies #Music and Audio Processing #Neuroscience and Music Perception #Sound (cs.SD) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2001.01720

openalex publication_date 2020/01/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In recent years, artificial neural networks (ANNs) have become a universal tool for tackling real-world problems. ANNs have also shown great success in music-related tasks including music summarization and classification, similarity estimation, computer-aided or autonomous composition, and automatic music analysis. As structure is a fundamental characteristic of Western music, it plays a role in all these tasks. Some structural aspects are particularly challenging to learn with current ANN architectures. This is especially true for mid- and high-level self-similarity, tonal and rhythmic relationships. In this thesis, I explore the application of ANNs to different aspects of musical structure modeling, identify some challenges involved and propose strategies to address them. First, using probability estimations of a Restricted Boltzmann Machine (RBM), a probabilistic bottom-up approach to melody segmentation is studied. Then, a top-down method for imposing a high-level structural template in music generation is presented, which combines Gibbs sampling using a convolutional RBM with gradient-descent optimization on the intermediate solutions. Furthermore, I motivate the relevance of musical transformations in structure modeling and show how a connectionist model, the Gated Autoencoder (GAE), can be employed to learn transformations between musical fragments. For learning transformations in sequences, I propose a special predictive training of the GAE, which yields a representation of polyphonic music as a sequence of intervals. Furthermore, the applicability of these interval representations to a top-down discovery of repeated musical sections is shown. Finally, a recurrent variant of the GAE is proposed, and its efficacy in music prediction and modeling of low-level repetition structure is demonstrated.

Citations

• Distilling the Knowledge in a Neural Network
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Long Short-Term Memory
• Training Products of Experts by Minimizing Contrastive Divergence
• A Fast Learning Algorithm for Deep Belief Nets
• The Smooth Signal Redundancy Hypothesis: A Functional Explanation for Relationships between Redundancy, Prosodic Prominence, and Duration in Spontaneous Speech
• Backpropagation Applied to Handwritten Zip Code Recognition
• Distributed Representations of Words and Phrases and their Compositionality
• Text-based LSTM networks for Automatic Music Composition
• Counterpoint By Convolution.
• A Predictive Model for Music Based on Learned Interval Representations
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Related

• Learning Musical Relations using Gated Autoencoders
• A Predictive Model for Music Based on Learned Interval Representations
• Deep Learning for Music
• Music Generation with Temporal Structure Augmentation
• Generating Music using an LSTM Network