Rationale-Enhanced Language Models are Better Continual Relation Learners

2023/10/10 by Weimin Xiong, Xiong, Weimin, Yifan Song +5 · 3 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2310.06547

openalex publication_date 2023/10/10 · openalex created_date 2023/10/12 · openalex updated_date 2026/07/28

Abstract

Continual relation extraction (CRE) aims to solve the problem of catastrophic forgetting when learning a sequence of newly emerging relations. Recent CRE studies have found that catastrophic forgetting arises from the model's lack of robustness against future analogous relations. To address the issue, we introduce rationale, i.e., the explanations of relation classification results generated by large language models (LLM), into CRE task. Specifically, we design the multi-task rationale tuning strategy to help the model learn current relations robustly. We also conduct contrastive rationale replay to further distinguish analogous relations. Experimental results on two standard benchmarks demonstrate that our method outperforms the state-of-the-art CRE models.

Cited by

• ADEPT: Continual Pretraining via Adaptive Expansion and Dynamic Decoupled Tuning
• SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs
• Learning Wisdom from Errors: Promoting LLM's Continual Relation Learning through Exploiting Error Cases

Related

• Learning Wisdom from Errors: Promoting LLM's Continual Relation Learning through Exploiting Error Cases
• Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation
• Preserving Generalization of Language models in Few-shot Continual Relation Extraction
• Post-Training Language Models for Continual Relation Extraction
• Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach