2022/11/26 by Zhong Ji, Ji, Zhong, Junhua Hu +7 · 2 citations
Computer Science · Mathematics · #Advanced Image and Video Retrieval Techniques #Artificial intelligence #Benchmark (surveying) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #FOS: Computer and information sciences #Information retrieval #Matching (statistics) #Mathematics #Modal #Multimodal Machine Learning Applications #Natural language processing #Networking and Internet Architecture (cs.NI) #Pattern recognition (psychology) #Phrase #Scale (ratio) #Sentence #Task (project management) #Transformer #Video Surveillance and Tracking Methods
paper · pdf · doi:10.48550/arxiv.2212.11958
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2022/11/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Text-based person search (TBPS) is of significant importance in intelligent surveillance, which aims to retrieve pedestrian images with high semantic relevance to a given text description. This retrieval task is characterized with both modal heterogeneity and fine-grained matching. To implement this task, one needs to extract multi-scale features from both image and text domains, and then perform the cross-modal alignment. However, most existing approaches only consider the alignment confined at their individual scales, e.g., an image-sentence or a region-phrase scale. Such a strategy adopts the presumable alignment in feature extraction, while overlooking the cross-scale alignment, e.g., image-phrase. In this paper, we present a transformer-based model to extract multi-scale representations, and perform Asymmetric Cross-Scale Alignment (ACSA) to precisely align the two modalities. Specifically, ACSA consists of a global-level alignment module and an asymmetric cross-attention module, where the former aligns an image and texts on a global scale, and the latter applies the cross-attention mechanism to dynamically align the cross-modal entities in region/image-phrase scales. Extensive experiments on two benchmark datasets CUHK-PEDES and RSTPReid demonstrate the effectiveness of our approach. Codes are available at \hrefurlhttps://github.com/mul-hjh/ACSA.