Improving Event Duration Prediction via Time-aware Pre-training

2020/11/05 by Zonglin Yang, Xinya Du, Yang, Zonglin +5
Computer Science · #Advanced Text Analysis Techniques #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2011.02610

openalex publication_date 2020/11/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

End-to-end models in NLP rarely encode external world knowledge about length of time. We introduce two effective models for duration prediction, which incorporate external knowledge by reading temporal-related news sentences (time-aware pre-training). Specifically, one model predicts the range/unit where the duration value falls in (R-pred); and the other predicts the exact duration value E-pred. Our best model -- E-pred, substantially outperforms previous work, and captures duration information more accurately than R-pred. We also demonstrate our models are capable of duration prediction in the unsupervised setting, outperforming the baselines.

Citations

• Teaching Machines to Read and Comprehend
• Long Short-Term Memory
• Verbs and Times

Related

• On Optimality and Human Prediction of Event Duration in Real-Time, Real-World Contexts
• Time-Dependent Representation for Neural Event Sequence Prediction
• BiTimeBERT: Extending Pre-Trained Language Representations with Bi-Temporal Information
• Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Models
• ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning