CORL: Research-oriented Deep Offline Reinforcement Learning Library

2022/10/13 by Denis Tarasov, Alexander Nikulin, Tarasov, Denis +7 · 36 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mobile Crowdsensing and Crowdsourcing #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2210.07105

openalex publication_date 2022/10/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

CORL is an open-source library that provides thoroughly benchmarked single-file implementations of both deep offline and offline-to-online reinforcement learning algorithms. It emphasizes a simple developing experience with a straightforward codebase and a modern analysis tracking tool. In CORL, we isolate methods implementation into separate single files, making performance-relevant details easier to recognize. Additionally, an experiment tracking feature is available to help log metrics, hyperparameters, dependencies, and more to the cloud. Finally, we have ensured the reliability of the implementations by benchmarking commonly employed D4RL datasets providing a transparent source of results that can be reused for robust evaluation tools such as performance profiles, probability of improvement, or expected online performance.

Cited by

• Adaptive Replay Buffer for Offline-to-Online Reinforcement Learning
• PrivORL: Differentially Private Synthetic Dataset for Offline Reinforcement Learning
• Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning
• State Diversity Matters in Offline Behavior Distillation
• Long-Horizon Model-Based Offline Reinforcement Learning Without Explicit Conservatism
• Forecasting in Offline Reinforcement Learning for Non-stationary Environments
• One-Step Generative Policies with Q-Learning: A Reformulation of MeanFlow
• A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms
• Quantile Q-Learning: Revisiting Offline Extreme Q-Learning with Quantile Regression
• From Static Constraints to Dynamic Adaptation: Sample-Level Constraint Relaxation for Offline-to-Online Reinforcement Learning
• Expert or not? assessing data quality in offline reinforcement learning
• Adjusting the Output of Decision Transformer with Action Gradient
• Unleashing Flow Policies with Distributional Critics
• Adaptive Scaling of Policy Constraints for Offline Reinforcement Learning
• HALO: Human Preference Aligned Offline Reward Learning for Robot Navigation
• Neuro-Inspired Inverse Learning for Planning and Control
• Policy-Based Trajectory Clustering in Offline Reinforcement Learning
• Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
• Intention-Conditioned Flow Occupancy Models
• Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Data
• Flow-Based Single-Step Completion for Efficient and Expressive Policy Learning
• Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity
• Is Optimal Transport Necessary for Inverse Reinforcement Learning?
• Horizon Reduction Makes RL Scalable
• Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization
• Scaling Offline RL via Efficient and Expressive Shortcut Models
• Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL
• CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zero
• PyTupli: A Scalable Infrastructure for Collaborative Offline Reinforcement Learning Projects
• Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learning
• ReBRAC-v2: The Return of the King
• Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Model
• Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition
• Efficient and Uncertainty-Aware Diffusion Framework for Offline-to-Online Reinforcement Learning
• Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learning
• A Clean Slate for Offline Reinforcement Learning

Related

• CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms
• NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios
• Improving and Benchmarking Offline Reinforcement Learning Algorithms
• D4RL: Datasets for Deep Data-Driven Reinforcement Learning
• NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning