2021/09/30 by Satya Almasian, Dennis Aumiller, Almasian, Satya +3 · 1 citation
Computer Science · Engineering · #Architecture #Artificial intelligence #Artificial neural network #Computation and Language (cs.CL) #Computer science #Encoder #Engineering #FOS: Computer and information sciences #Language model #Machine learning #Natural Language Processing Techniques #Natural language #Natural language processing #Security token #Speech and dialogue systems #Topic Modeling #Transformer #cs.CL
paper · pdf · doi:10.48550/arxiv.2109.14927
published in arXiv (Cornell University) (Cornell University) · unreliable evaluation results for Seq2seq models
openalex publication_date 2021/09/30 · arxiv created 2022/01/24 · arxiv updated 2022/01/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Temporal expressions in text play a significant role in language understanding and correctly identifying them is fundamental to various retrieval and natural language processing systems. Previous works have slowly shifted from rule-based to neural architectures, capable of tagging expressions with higher accuracy. However, neural models can not yet distinguish between different expression types at the same level as their rule-based counterparts. In this work, we aim to identify the most suitable transformer architecture for joint temporal tagging and type classification, as well as, investigating the effect of semi-supervised training on the performance of these systems. Based on our study of token classification variants and encoder-decoder architectures, we present a transformer encoder-decoder model using the RoBERTa language model as our best performing system. By supplementing training resources with weakly labeled data from rule-based systems, our model surpasses previous works in temporal tagging and type classification, especially on rare classes. Our code and pre-trained experiments are available at: https://github.com/satya77/TransformerTemporalTagger