EvoEdit: Lifelong Free-Text Knowledge Editing through Latent Perturbation Augmentation and Knowledge-driven Parameter Fusion

2025/12/04 by Pengfei Cao, Cao, Pengfei, Zhongxia Ji +7
Computer Science · #Advanced Graph Neural Networks #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2512.04545

openalex publication_date 2025/12/04 · openalex created_date 2025/12/06 · openalex updated_date 2026/07/28

Abstract

Adjusting the outdated knowledge of large language models (LLMs) after deployment remains a major challenge. This difficulty has spurred the development of knowledge editing, which seeks to accurately and efficiently modify a model's internal (parametric) knowledge without retraining it from scratch. However, existing methods suffer from two limitations. First, they depend on structured triplets that are misaligned with the free-text nature of LLM pretraining and fail to capture the nuanced relationships among facts. Second, they typically support one-time knowledge updates, with relatively limited research on the problem of sequential or lifelong editing. To address these gaps, we propose a new task, Lifelong Free-text Knowledge Editing (LF-Edit), which enables models to incorporate updates expressed in natural language and supports continual editing over time. Despite its promise, LF-Edit faces the dual challenge of integrating new knowledge while mitigating the forgetting of prior information. To foster research on this new task, we construct a large-scale benchmark, Multi-Rank Lifelong Free-text Editing Benchmark (MRLF-Bench), containing 16,835 free-text edit requests. We further design a cognitively inspired multi-rank evaluation framework encompassing four levels: memorization, understanding, constrained comprehension, and reasoning. To tackle the challenges inherent in LF-Edit, we introduce a novel approach named EvoEdit that enhances knowledge injection through Latent Perturbation Augmentation and preserves prior information via Knowledge-driven Parameter Fusion. Experimental results demonstrate that EvoEdit substantially outperforms existing knowledge editing methods on the proposed LF-Edit task.

Citations

• Synergizing RAG and Reasoning: A Systematic Review
• Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs
• AnyEdit: Edit Any Knowledge Encoded in Language Models
• Parameter-Efficient Fine-Tuning for Foundation Models
• One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models
• MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic
• MEMLA: Enhancing Multilingual Knowledge Editing with Neuron-Masked Low-Rank Adaptation
• WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models
• EMR-Merging: Tuning-Free High-Performance Model Merging
• Localizing Task Information for Improved Model Merging and Compression
• Checkpoint Merging via Bayesian Optimization in LLM Pretraining
• Evolutionary optimization of model merging recipes
• Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing
• Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models
• Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models
• WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing
• A Comprehensive Study of Knowledge Editing for Large Language Models
• Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
• Massive Editing for Large Language Models via Meta Learning
• Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
• Can We Edit Multimodal Large Language Models?
• AdaMerging: Adaptive Model Merging for Multi-Task Learning
• MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
• Can We Edit Factual Knowledge by In-Context Learning?
• Editing Large Language Models: Problems, Methods, and Opportunities
• Dataless Knowledge Fusion by Merging Weights of Language Models
• Editing Models with Task Arithmetic
• Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
• Mass-Editing Memory in a Transformer
• Memory-Based Model Editing at Scale
• A Survey on Neural Open Information Extraction: Current Status and Future Directions
• Locating and Editing Factual Associations in GPT
• Merging Models with Fisher-Weighted Averaging
• Fast Model Editing at Scale
• Editing Factual Knowledge in Language Models
• Zero-Shot Relation Extraction via Reading Comprehension

Related

• EtCon: Edit-then-Consolidate for Reliable Knowledge Editing
• Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning
• EvoEdit: Evolving Null-space Alignment for Robust and Efficient Knowledge Editing
• WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs
• Learning to Edit: Aligning LLMs with Knowledge Editing