2020/07/21 by Vincent Lostanlen, Christian El-Hajj, Lostanlen, Vincent +9 · 1 citation
Arts and Humanities · Computer Science · Neuroscience · #Audio and Speech Processing (eess.AS) #Diverse Musicological Studies #FOS: Computer and information sciences #FOS: Electrical engineering #Hearing Loss and Rehabilitation #Machine Learning (cs.LG) #Music and Audio Processing #Sound (cs.SD) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2007.10926
openalex publication_date 2020/07/21 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Instrumental playing techniques such as vibratos, glissandos, and trills\noften denote musical expressivity, both in classical and folk contexts.\nHowever, most existing approaches to music similarity retrieval fail to\ndescribe timbre beyond the so-called "ordinary" technique, use instrument\nidentity as a proxy for timbre quality, and do not allow for customization to\nthe perceptual idiosyncrasies of a new subject. In this article, we ask 31\nhuman subjects to organize 78 isolated notes into a set of timbre clusters.\nAnalyzing their responses suggests that timbre perception operates within a\nmore flexible taxonomy than those provided by instruments or playing techniques\nalone. In addition, we propose a machine listening model to recover the cluster\ngraph of auditory similarities across instruments, mutes, and techniques. Our\nmodel relies on joint time--frequency scattering features to extract\nspectrotemporal modulations as acoustic features. Furthermore, it minimizes\ntriplet loss in the cluster graph by means of the large-margin nearest neighbor\n(LMNN) metric learning algorithm. Over a dataset of 9346 isolated notes, we\nreport a state-of-the-art average precision at rank five (AP@5) of\n99.0 %\±1. An ablation study demonstrates that removing either the joint\ntime--frequency scattering transform or the metric learning algorithm\nnoticeably degrades performance.\n