Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learning

2025/04/24 by Mingqi Yuan, Qi Wang, Yuan, Mingqi +17 · 1 citation
Neuroscience · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neuroscience and Neural Engineering

paper · pdf · doi:10.48550/arxiv.2504.17490

openalex publication_date 2025/04/24 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

Developing lifelong learning agents is crucial for artificial general intelligence (AGI). However, deep reinforcement learning (RL) systems often suffer from plasticity loss, where neural networks gradually lose their ability to adapt during training. Despite its significance, this field lacks unified benchmarks and evaluation protocols. We introduce Plasticine, the first open-source framework for benchmarking plasticity optimization in deep RL. Plasticine provides single-file implementations of over 13 mitigation methods, 6 evaluation metrics, and learning scenarios with increasing non-stationarity levels from standard to continually varying environments. This framework enables researchers to systematically quantify plasticity loss, evaluate mitigation strategies, and analyze plasticity dynamics across different contexts. Our documentation, examples, and source code are available at https://github.com/RLE-Foundation/Plasticine.

Citations

• Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning
• Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn
• Hyperspherical Normalization for Scalable Deep Reinforcement Learning
• Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Loss
• Parseval Regularization for Continual Reinforcement Learning
• Plasticity Loss in Deep Reinforcement Learning: A Survey
• Plastic Learning with Deep Fourier Features
• SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning
• Simplifying Deep Temporal Difference Learning
• Normalization and effective learning rates in reinforcement learning
• A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning
• Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learning
• Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control
• Disentangling the Causes of Plasticity Loss in Neural Networks
• Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning
• Discounted Adaptive Online Learning: Towards Better Regularization
• Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages
• Small batch deep reinforcement learning
• Maintaining Plasticity in Continual Learning via Regenerative Regularization
• A Definition of Continual Reinforcement Learning
• PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning
• Deep Reinforcement Learning with Plasticity Injection
• Loss of Plasticity in Continual Deep Reinforcement Learning
• Understanding plasticity in neural networks
• The Dormant Neuron Phenomenon in Deep Reinforcement Learning
• CORL: Research-oriented Deep Offline Reinforcement Learning Library
• The Primacy Bias in Deep Reinforcement Learning
• Understanding and Preventing Capacity Loss in Reinforcement Learning
• DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
• Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning
• Leveraging Procedural Generation to Benchmark Reinforcement Learning
• Addressing Function Approximation Error in Actor-Critic Methods
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• DeepMind Control Suite
• Proximal Policy Optimization Algorithms
• A Distributional Perspective on Reinforcement Learning
• Layer Normalization
• High-Dimensional Continuous Control Using Generalized Advantage\n Estimation
• The Arcade Learning Environment: An Evaluation Platform for General Agents

Cited by

• Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learning

Related

• Deep Reinforcement Learning with Plasticity Injection
• Plasticity Loss in Deep Reinforcement Learning: A Survey
• PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning
• Understanding plasticity in neural networks
• A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning