2025/06/26 by Prajwal Koirala, Koirala, Prajwal, Cody Fleming +1 · 5 citations
Computer Science · #Data Stream Mining Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data #Recommender Systems and Techniques #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2506.21427
openalex publication_date 2025/06/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Generative models such as diffusion and flow-matching offer expressive policies for offline reinforcement learning (RL) by capturing rich, multimodal action distributions, but their iterative sampling introduces high inference costs and training instability due to gradient propagation across sampling steps. We propose the Single-Step Completion Policy (SSCP), a generative policy trained with an augmented flow-matching objective to predict direct completion vectors from intermediate flow samples, enabling accurate, one-shot action generation. In an off-policy actor-critic framework, SSCP combines the expressiveness of generative models with the training and inference efficiency of unimodal policies, without requiring long backpropagation chains. Our method scales effectively to offline, offline-to-online, and online RL settings, offering substantial gains in speed and adaptability over diffusion-based baselines. We further extend SSCP to goal-conditioned RL, enabling flat policies to exploit subgoal structures without explicit hierarchical inference. SSCP achieves strong results across standard offline RL and behavior cloning benchmarks, positioning it as a versatile, expressive, and efficient framework for deep RL and sequential decision-making. The code is available at https://github.com/PrajwalKoirala/SSCP-Single-Step-Completion-Policy.