Detecting Model Drifts in Non-Stationary Environment Using Edit Operation Measures

2025/09/14 by Lee, Chang-Hwan, Shim, Alexander
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2509.11367

Abstract

Reinforcement learning (RL) agents typically assume stationary environment dynamics. Yet in real-world applications such as healthcare, robotics, and finance, transition probabilities or reward functions may evolve, leading to model drift. This paper proposes a novel framework to detect such drifts by analyzing the distributional changes in sequences of agent behavior. Specifically, we introduce a suite of edit operation-based measures to quantify deviations between state-action trajectories generated under stationary and perturbed conditions. Our experiments demonstrate that these measures can effectively distinguish drifted from non-drifted scenarios, even under varying levels of noise, providing a practical tool for drift detection in non-stationary RL environments.

Citations

• Offline Reinforcement Learning from Datasets with Structured Non-Stationarity
• A Behavior-Aware Approach for Deep Reinforcement Learning in Non-stationary Environments without Known Change Points
• Sample Complexity Characterization for Linear Contextual MDPs
• Assessing the Impact of Distribution Shift on Reinforcement Learning Performance
• Online Reinforcement Learning in Non-Stationary Context-Driven Environments
• A survey on machine learning for recurring concept drifting data streams
• Offline RL Policies Should be Trained to be Adaptive
• Testing Stationarity and Change Point Detection in Reinforcement Learning
• Doubly Robust Off-policy Value Evaluation for Reinforcement Learning
• Dynamic programming algorithm optimization for spoken word recognition

Related

• DriftSurf: A Risk-competitive Learning Algorithm under Concept Drift
• Non-stochastic Bandits With Evolving Observations
• Non-Stationary Reinforcement Learning: The Blessing of (More) Optimism
• Detecting Interpretable Subgroup Drifts
• A Behavior-Aware Approach for Deep Reinforcement Learning in Non-stationary Environments without Known Change Points