I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch

2020/12/08 by Joseph Turian, Turian, Joseph, Max Henry +1 · 4 citations
Computer Science · Neuroscience · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Hearing Loss and Rehabilitation #Machine Learning (cs.LG) #Music Technology and Sound Studies #Music and Audio Processing #Sound (cs.SD) #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2012.04572

openalex publication_date 2020/12/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Growing research demonstrates that synthetic failure modes imply poor generalization. We compare commonly used audio-to-audio losses on a synthetic benchmark, measuring the pitch distance between two stationary sinusoids. The results are surprising: many have poor sense of pitch direction. These shortcomings are exposed using simple rank assumptions. Our task is trivial for humans but difficult for these audio distances, suggesting significant progress can be made in self-supervised audio learning by improving current losses.

Cited by

• Reverse Engineering of Music Mixing Graphs with Differentiable Processors and Iterative Pruning
• Assessing the Alignment of Audio Representations with Timbre Similarity Ratings
• Evaluating Sound Similarity Metrics for Differentiable, Iterative Sound-Matching
• Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching

Related

• Fr 'echet Audio Distance: A Metric for Evaluating Music Enhancement\n Algorithms
• Pitch Contour Exploration Across Audio Domains: A Vision-Based Transfer Learning Approach
• DEEPF0: End-To-End Fundamental Frequency Estimation for Music and Speech Signals
• Mel Spectrogram Inversion with Stable Pitch
• On loss functions and evaluation metrics for music source separation