2020/03/27 by Michael A. Lepori, Chaz Firestone, Lepori, Michael A +2
Computer Science · Neuroscience · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Audio and Speech Processing (eess.AS) #Cognitive Science and Education Research #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2003.12362
openalex publication_date 2020/03/27 · openalex created_date 2022/10/03 · openalex updated_date 2026/07/28
The rise of machine-learning systems that process sensory input has brought\nwith it a rise in comparisons between human and machine perception. But such\ncomparisons face a challenge: Whereas machine perception of some stimulus can\noften be probed through direct and explicit measures, much of human perceptual\nknowledge is latent, incomplete, or unavailable for explicit report. Here, we\nexplore how this asymmetry can cause such comparisons to misestimate the\noverlap in human and machine perception. As a case study, we consider human\nperception of \adversarial speech -- synthetic audio commands that are\nrecognized as valid messages by automated speech-recognition systems but that\nhuman listeners reportedly hear as meaningless noise. In five experiments, we\nadapt task designs from the human psychophysics literature to show that even\nwhen subjects cannot freely transcribe such speech commands (the previous\nbenchmark for human understanding), they often can demonstrate other forms of\nunderstanding, including discriminating adversarial speech from closely matched\nnon-speech (Experiments 1--2), finishing common phrases begun in adversarial\nspeech (Experiments 3--4), and solving simple math problems posed in\nadversarial speech (Experiment 5) -- even for stimuli previously described as\nunintelligible to human listeners. We recommend the adoption of such "sensitive\ntests" when comparing human and machine perception, and we discuss the broader\nconsequences of such approaches for assessing the overlap between systems.\n