2018/12/25 by Lin Sun, Kui Jia, Sun, Lin +10 · 1 citation
Computer Science · #Anomaly Detection Techniques and Applications #Artificial intelligence #Artificial neural network #Benchmark (surveying) #Block (permutation group theory) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Exploit #FOS: Computer and information sciences #Feature (linguistics) #Hand Gesture Recognition Systems #Human Pose and Action Recognition #Key (lock) #Machine Learning (cs.LG) #Machine learning #Pattern recognition (psychology) #Reciprocal #Recurrent neural network #Stack (abstract data type) #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.1812.10071
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2018/12/25 · arxiv created 2019/03/25 · arxiv updated 2019/03/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Many semantic video analysis tasks can benefit from multiple, heterogenous signals. For example, in addition to the original RGB input sequences, sequences of optical flow are usually used to boost the performance of human action recognition in videos. To learn from these heterogenous input sources, existing methods reply on two-stream architectural designs that contain independent, parallel streams of Recurrent Neural Networks (RNNs). However, two-stream RNNs do not fully exploit the reciprocal information contained in the multiple signals, let alone exploit it in a recurrent manner. To this end, we propose in this paper a novel recurrent architecture, termed Coupled Recurrent Network (CRN), to deal with multiple input sources. In CRN, the parallel streams of RNNs are coupled together. Key design of CRN is a Recurrent Interpretation Block (RIB) that supports learning of reciprocal feature representations from multiple signals in a recurrent manner. Different from RNNs which stack the training loss at each time step or the last time step, we propose an effective and efficient training strategy for CRN. Experiments show the efficacy of the proposed CRN. In particular, we achieve the new state of the art on the benchmark datasets of human action recognition and multi-person pose estimation.