2018/10/09 by Loris Bazzani, Tobias Domhan, Bazzani, Loris +3 · 1 citation
Computer Science · Engineering · #Artificial intelligence #Biology #Closed captioning #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Engineering #FOS: Computer and information sciences #Human Pose and Action Recognition #Image (mathematics) #Machine translation #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #Speech recognition #Task (project management) #Translation (biology) #cs.CV
paper · pdf · doi:10.48550/arxiv.1810.04101
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2018/10/09 · arxiv created 2018/10/15 · arxiv updated 2018/10/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Image captioning is an interdisciplinary research problem that stands between computer vision and natural language processing. The task is to generate a textual description of the content of an image. The typical model used for image captioning is an encoder-decoder deep network, where the encoder captures the essence of an image while the decoder is responsible for generating a sentence describing the image. Attention mechanisms can be used to automatically focus the decoder on parts of the image which are relevant to predict the next word. In this paper, we explore different decoders and attentional models popular in neural machine translation, namely attentional recurrent neural networks, self-attentional transformers, and fully-convolutional networks, which represent the current state of the art of neural machine translation. The image captioning module is available as part of SOCKEYE at https://github.com/awslabs/sockeye which tutorial can be found at https://awslabs.github.io/sockeye/imagecaptioning.html .