2021/06/18 by Carmelo Scribano, Scribano, Carmelo, Davide Sapienza +7
Computer Science · Social Sciences · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Mobility and Location-Based Analysis #Multimodal Machine Learning Applications #Video Surveillance and Tracking Methods
paper · pdf · doi:10.48550/arxiv.2106.10153
openalex publication_date 2021/06/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Combining Natural Language with Vision represents a unique and interesting\nchallenge in the domain of Artificial Intelligence. The AI City Challenge Track\n5 for Natural Language-Based Vehicle Retrieval focuses on the problem of\ncombining visual and textual information, applied to a smart-city use case. In\nthis paper, we present All You Can Embed (AYCE), a modular solution to\ncorrelate single-vehicle tracking sequences with natural language. The main\nbuilding blocks of the proposed architecture are (i) BERT to provide an\nembedding of the textual descriptions, (ii) a convolutional backbone along with\na Transformer model to embed the visual information. For the training of the\nretrieval model, a variation of the Triplet Margin Loss is proposed to learn a\ndistance measure between the visual and language embeddings. The code is\npublicly available at https://github.com/cscribano/AYCE2021.\n