2020/09/24 by Daiki Takeuchi, Yuma Koizumi, Takeuchi, Daiki +7 · 1 citation
Arts and Humanities · Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Sound (cs.SD) #Speech and dialogue systems #Subtitles and Audiovisual Media #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2009.11436
openalex publication_date 2020/09/24 · openalex created_date 2020/10/01 · openalex updated_date 2026/07/28
The system we used for Task 6 (Automated Audio Captioning)of the Detection and Classification of Acoustic Scenes and Events(DCASE) 2020 Challenge combines three elements, namely, dataaugmentation, multi-task learning, and post-processing, for audiocaptioning. The system received the highest evaluation scores, butwhich of the individual elements most fully contributed to its perfor-mance has not yet been clarified. Here, to asses their contributions,we first conducted an element-wise ablation study on our systemto estimate to what extent each element is effective. We then con-ducted a detailed module-wise ablation study to further clarify thekey processing modules for improving accuracy. The results showthat data augmentation and post-processing significantly improvethe score in our system. In particular, mix-up data augmentationand beam search in post-processing improve SPIDEr by 0.8 and 1.6points, respectively.