ILCL: Inverse Logic-Constraint Learning from Temporally Constrained Demonstrations

2025/07/15 by Minwoo Cho, Cho, Minwoo, Jaehwi Jang +3 · 1 citation
Computer Science · #Semantic Web and Ontologies #Natural Language Processing Techniques #Logic, Reasoning, and Knowledge

paper · pdf · doi:10.48550/arxiv.2507.11000

Abstract

We aim to solve the problem of temporal-constraint learning from demonstrations to reproduce demonstration-like logic-constrained behaviors. Learning logic constraints is challenging due to the combinatorially large space of possible specifications and the ill-posed nature of non-Markovian constraints. To figure it out, we introduce a novel temporal-constraint learning method, which we call inverse logic-constraint learning (ILCL). Our method frames ICL as a two-player zero-sum game between 1) a genetic algorithm-based temporal-logic mining (GA-TL-Mining) and 2) logic-constrained reinforcement learning (Logic-CRL). GA-TL-Mining efficiently constructs syntax trees for parameterized truncated linear temporal logic (TLTL) without predefined templates. Subsequently, Logic-CRL finds a policy that maximizes task rewards under the constructed TLTL constraints via a novel constraint redistribution scheme. Our evaluations show ILCL outperforms state-of-the-art baselines in learning and transferring TL constraints on four temporally constrained tasks. We also demonstrate successful transfer to real-world peg-in-shallow-hole tasks.

Citations

• A Unified Approach to Multi-task Legged Navigation: Temporal Logic Meets Reinforcement Learning
• LTL-Constrained Policy Optimization with Cycle Experience Replay
• Interpretable Imitation Learning via Generative Adversarial STL Inference and Control
• Learning Safety Constraints From Demonstration Using One-Class Decision Trees
• Learning Shared Safety Constraints from Multi-task Demonstrations
• Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Control
• Learning Signal Temporal Logic through Neural Network for Interpretable Classification
• Differentiable Rule Induction with Learned Relational Features
• Adversarial Option-Aware Hierarchical Imitation Learning
• Inverse Constrained Reinforcement Learning
• Learning Guidance Rewards with Trajectory-space Smoothing
• Learning to Walk in the Real World with Minimal Human Effort
• Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning
• Interpretable Classification of Time-Series Data using Efficient\n Enumerative Techniques
• RUDDER: Return Decomposition for Delayed Rewards
• Reward Constrained Policy Optimization
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• A Robust Genetic Algorithm for Learning Temporal Specifications from Data
• Constrained Policy Optimization
• Reinforcement Learning With Temporal Logic Rewards
• Correct-by-synthesis reinforcement learning with temporal logic constraints

Cited by

• A Visuo-Tactile Data Collection System with Haptic Feedback for Coarse-to-Fine Imitation Learning

Related

• Constrained LTL Specification Learning from Examples
• Explaining Multi-stage Tasks by Learning Temporal Logic Formulas from Suboptimal Demonstrations
• DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RL
• Learning temporal formulas from examples is hard
• Integrating Explanations in Learning LTL Specifications from Demonstrations