2017/11/21 by Yi Tay, Tay, Yi, Luu Anh Tuan +3
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #Expert finding and Q&A systems #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Topic Modeling #cs.AI #cs.CL #cs.IR
paper · pdf · doi:10.48550/arxiv.1711.07656
Accepted to AAAI2018
arxiv created 2017/11/21 · openalex publication_date 2017/11/21 · arxiv updated 2017/11/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Temporal gates play a significant role in modern recurrent-based neural encoders, enabling fine-grained control over recursive compositional operations over time. In recurrent models such as the long short-term memory (LSTM), temporal gates control the amount of information retained or discarded over time, not only playing an important role in influencing the learned representations but also serving as a protection against vanishing gradients. This paper explores the idea of learning temporal gates for sequence pairs (question and answer), jointly influencing the learned representations in a pairwise manner. In our approach, temporal gates are learned via 1D convolutional layers and then subsequently cross applied across question and answer for joint learning. Empirically, we show that this conceptually simple sharing of temporal gates can lead to competitive performance across multiple benchmarks. Intuitively, what our network achieves can be interpreted as learning representations of question and answer pairs that are aware of what each other is remembering or forgetting, i.e., pairwise temporal gating. Via extensive experiments, we show that our proposed model achieves state-of-the-art performance on two community-based QA datasets and competitive performance on one factoid-based QA dataset.