2020/10/19 by Kin Wai Cheuk, Yin-Jyun Luo, Cheuk, Kin Wai +5
Arts and Humanities · Computer Science · #Audio and Speech Processing (eess.AS) #Diverse Musicological Studies #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Music Technology and Sound Studies #Music and Audio Processing #Sound (cs.SD) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2010.09969
openalex publication_date 2020/10/19 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Most of the state-of-the-art automatic music transcription (AMT) models break\ndown the main transcription task into sub-tasks such as onset prediction and\noffset prediction and train them with onset and offset labels. These\npredictions are then concatenated together and used as the input to train\nanother model with the pitch labels to obtain the final transcription. We\nattempt to use only the pitch labels (together with spectrogram reconstruction\nloss) and explore how far this model can go without introducing supervised\nsub-tasks. In this paper, we do not aim at achieving state-of-the-art\ntranscription accuracy, instead, we explore the effect that spectrogram\nreconstruction has on our AMT model. Our proposed model consists of two U-nets:\nthe first U-net transcribes the spectrogram into a posteriorgram, and a second\nU-net transforms the posteriorgram back into a spectrogram. A reconstruction\nloss is applied between the original spectrogram and the reconstructed\nspectrogram to constrain the second U-net to focus only on reconstruction. We\ntrain our model on three different datasets: MAPS, MAESTRO, and MusicNet. Our\nexperiments show that adding the reconstruction loss can generally improve the\nnote-level transcription accuracy when compared to the same model without the\nreconstruction part. Moreover, it can also boost the frame-level precision to\nbe higher than the state-of-the-art models. The feature maps learned by our\nU-net contain gridlike structures (not present in the baseline model) which\nimplies that with the presence of the reconstruction loss, the model is\nprobably trying to count along both the time and frequency axis, resulting in a\nhigher note-level transcription accuracy.\n