2020/01/13 by Seth H. Huang, Huang, Seth H., Xu Lingjie +5 · 1 citation
Computer Science · #Advanced Neural Network Applications #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.2001.04077
arxiv created 2020/01/13 · openalex publication_date 2020/01/13 · arxiv updated 2020/01/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We present a novel architecture, residual attention net (RAN), which merges a sequence architecture, universal transformer, and a computer vision architecture, residual net, with a high-way architecture for cross-domain sequence modeling. The architecture aims at addressing the long dependency issue often faced by recurrent-neural-net-based structures. This paper serves as a proof-of-concept for a new architecture, with RAN aiming at providing the model a higher level understanding of sequence patterns. To our best knowledge, we are the first to propose such an architecture. Out of the standard 85 UCR data sets, we have achieved 35 state-of-the-art results with 10 results matching current state-of-the-art results without further model fine-tuning. The results indicate that such architecture is promising in complex, long-sequence modeling and may have vast, cross-domain applications.