Editable Neural Networks

2020/04/01 by Anton Sinitsin, Sinitsin, Anton, Vsevolod Plokhotnyuk +7 · 5 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2004.00345

openalex publication_date 2020/04/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

These days deep neural networks are ubiquitously used in a wide range of tasks, from image classification and machine translation to face identification and self-driving cars. In many applications, a single model error can lead to devastating financial, reputational and even life-threatening consequences. Therefore, it is crucially important to correct model mistakes quickly as they appear. In this work, we investigate the problem of neural network editing - how one can efficiently patch a mistake of the model on a particular sample, without influencing the model behavior on other samples. Namely, we propose Editable Training, a model-agnostic training technique that encourages fast editing of the trained model. We empirically demonstrate the effectiveness of this method on large-scale image classification and machine translation tasks.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• Distilling the Knowledge in a Neural Network
• Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions.
• Catastrophic Forgetting, Rehearsal and Pseudorehearsal
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• fairseq: A Fast, Extensible Toolkit for Sequence Modeling

Cited by

• Uncovering and Mitigating Transient Blindness in Multimodal Model Editing
• MolEdit: Knowledge Editing for Multimodal Molecule Language Models
• EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models
• KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning
• A Dual-Axis Taxonomy of Knowledge Editing for LLMs: From Mechanisms to Functions

Related

• A Survey on Knowledge Editing of Neural Networks
• Fast Model Editing at Scale
• Transformer-Patcher: One Mistake worth One Neuron
• Memory-Based Model Editing at Scale
• Rank-One Editing of Encoder-Decoder Models