2023/02/09 by Alexander Mehta, Mehta, Alexander, Ritik Jalisatgi +1
Computer Science · Engineering · Neuroscience · #Audio and Speech Processing (eess.AS) #Automated Road and Building Extraction #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Human-Computer Interaction (cs.HC) #Sound (cs.SD) #Tactile and Sensory Interactions #Video Surveillance and Tracking Methods #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2303.13536
openalex publication_date 2023/02/09 · openalex created_date 2023/03/30 · openalex updated_date 2026/07/28
An estimated 253 million people have visual impairments. These visual impairments affect everyday lives, and limit their understanding of the outside world. This can pose a risk to health from falling or collisions. We propose a solution to this through quick and detailed communication of environmental spatial geometry through sound, providing the blind and visually impaired the ability to understand their spatial environment through sound technology. The model consists of fast object detection and 3D environmental mapping, which is communicated through a series of quick sound notes. These sound notes are at different frequencies, pitches, and arrangements in order to precisely communicate the depth and location of points within the environment. Sounds are communicated in the form of musical notes in order to be easily recognizable and distinguishable. A unique algorithm is used to segment objects, providing minimal accuracy loss and improvement from the normal O(n2 ) to O(n) (which is significant, as N in point clouds can often be in the range of 105 ). In testing, we achieved an R-value of 0.866 on detailed objects and an accuracy of 87.5% on an outdoor scene at night with large amounts of noise. We also provide a supplementary video demo of our system.